Quantitative and Automated Analysis in Molecular Pathology

CATHERINE CONWAY, PH.D.:  So my agenda today, really I want to start off with the definition of what is molecular pathology, what are people doing today within pathology, and how does digital pathology fit into this.  I really want to introduce the benefits of the pathology and specifically talk about the image analysis applications and give you a good overview of why people are using image analysis and what the benefits are, how it can be fully automated, and give you some used case examples of what people are doing today within their research using image analysis.

     So, to start off, I think you know we all know that pathology plays a very critical role in the decision making when it comes to counter diagnostics.  And we believe that around 70% of all of the current diagnostic decisions are based on pathology information.  But pathology has evolved very much of late, so there is a lot more focus not just on the traditional diagnostics, but also looking at prognostics and therapeutics, and this has all fallen into the arena of the pathologist.  And this involves much more involvement of proteomic and genomic testing’s at the present.

     In addition to the technology evolving and the science evolving, also the standards and the criteria for pathologists is constantly being raised and evaluated, so there is much more of an emphasis on trying to automate some of the more mundane tasks, and also trying to increase inter- and intra-observer variability.  And also more focus has been placed on recording the information so it’s better documentation, better record keeping, and pathology reviews. 

     So, really moving into what molecular pathology is, and really that is the study of molecules, mainly DNA, RNA, or protein in a disease state.  And today it’s mostly used for the diagnostics of the disease, but of course it’s also used to guide the prevention and the treatment of that disease.  Some very common applications of the molecular assays today, there is a lot of passed around inherent genetic disease to hopefully prevent, enable to prevent, these measures.  These tests can be anything from, for example, an APC for colorectal cancer tests or consultations.  There is also more detailed molecular tests to help monitor the response after the disease and from the treatment, and whether the disease has returned. 

An application there may be within the genius of the BCR ADL tests, and finally predictive tests to see how somebody is actually going to respond to a drug.  So this is moving into the personalized medicines, and we've heard a lot of information around personalized medicine in recent years.  We have a drug and a specific test for that drug.

     And a very, very famous example is, of course, HER2 in breast, but there are also applications like ALK in lung cancer or BRAF within melanomas.  So we are seeing more and more research being done in this area to try and evaluate the personalized medicines and potential antibodies as part of that molecular test.

     So, molecular applications today, I have already mentioned that they can be either DNA, RNA, or protein, and more recently we are seeing combinations, so we have seen actually it’s better with both RNA and protein on one slide, so you are looking at immunohistochemistry and RNA ACE technology on a single slide.  The consideration for these assays are also how are they going to re-visualize or what type of test are you running or what types of conditions are you running it under. 

So, chromogenic or fluorescence would be typically the two options.  And there are pros and cons for each, but with each of these types, whether it’s fluorescent or chromogenic, you have to be able to visualize and actually see the test.  And we are going to speak more about that, the pros and cons of fluorescent and chromogenic a little bit later on.

So, the digital pathology; really what that is, is trying to bring digital images into an environment that is fully digital, so you capture the images, but the images alone are not enough.  We have to have software and an ability to manage these large datasets and also all of the data that is generated with these digital slides.  So, today it is used very much as an assist in treatment paths and review and research.  And we see it taking a leading role in the development of personalized medicine. 

There are many benefits of digital pathology, and where it really developed initially was around teaching and teaching applications, but today the benefits have grown more than that.  So it’s very much about getting access to expertise in real-time or as fast as possible.  So I think everyone on the call will understand the time lost and the difficulties in shipping slides from the patient to a location to try and access the best pathologists in the world.  So using digital pathology, it’s a lot easier to share digital slides rather than posting it to a remote location for fear of breakage or lost slides, and more importantly, the time taken to ship the slides. 

There are some efficiencies in the workflow,when we have to recall slides or when you have to go and actually pull old cases to try and take them, to actually go and grab the physical slide.  Using digital pathology is very easy to have that digital slide sitting in with your information in an electronic environment.  And remote access is something we see constantly growing and growing, so the pathologists are working remotely.  People and experts are in remote locations, so having the ability to share these slides into remote locations is so important and is one of the biggest advantages of digital pathology. 

For me, the image analysis is extremely important, and we are going to speak at length about image analysis for the benefits, but really I want to call it out here because it is one of the benefits that you can do automated image analysis to try to bring more information than is currently possible when looking down a microscope.  Image analysis can help with decision support, so we see very much, for example, the post-docs and researchers are doing image analysis or trying to review cases.  They don’t have access to pathologists every minute of every day, but if you were to use image analysis and train up to what your pathologist is reviewing, you can then use image analysis as a decision support just too, in your own research and in your own studies. 

And, finally, it is a wonderful way to integrate the data and manage the data, so you can pull up a digital slide in years to come and the data is the same as it was the day that it was digitized.  So it’s a centralized way of managing information, with easy access of that information, and to have it all in one place with all of the data that pertains to that patient or the case or the specimen that you are looking at.  So really with digital pathology, what we are trying to do, is enable the pathologists to do their job better and enable the researchers to do their job better, and we are trying to do that by making efficiencies in areas that under the microscope today may not be possible.

So, the applications of digital pathology, there are many applications, but the way I break this down really is if you want to share the slides, record, or action and the slides, and we can really incorporate your timeline as well as how easy it is to adopt compared to how much benefit it is to the actual patients or the specimens here.  So, for example, digital pathology, we started off with teaching because it was a novel way to share unique and rare cases to a class when they were getting their education and that is where we started with the software many years ago. 

And then that also evolved into continued professional development, so people getting accreditation, getting their hours of training by using digital pathology to review cases and get their skill sets enhanced.  We see a lot of used cases of digital pathology in tumor boards.  Again, it’s so much easier to share an image across multiple monitors than it is to look down a microscope with multiple people, so just again the ease of use.  And all of this team sharing, you are sharing information and you are looking up slides. 

The next benefits are the applications that are integrated into the electronic health record, so having your image or your digital image right there with all of the information that you need to make a decision.  And, again, you can - - it, with digital pathology you can make notes on the slide so when you have to go back and refer to a case, all of that information is recorded.  And it’s much easier to see it when you are looking at the slide than looking at a paper or your notes.  That information can live in the digital slide.  The action is really where it gets extremely interesting.  We see digital pathology, again, it can be used for second opinions in certain areas where you can send a slide to someone and gather their expertise and their opinion, and get that advice in real time. 

We also then of course see the advantages of digital pathology with image analysis and qualitative analysis that we are going to speak about today.  And, finally, in some countries, not in all countries, but there is always the option to do a primary diagnosis on digital pathology images in certain territories.  So you can see that there are many, many different applications with digital pathology today.

What I find particularly interesting is how digital pathology really has evolved over the years, so if we think about it from time from left to right and we see the path that has led us to digital pathology today, so you know we started off with microscopes and they are fantastic at visualization.  We get a great image.  When you are looking at a static image in a small area, that is hard to archive and that you can't share remotely.  So the next iteration of microscope was just very simply putting a camera on top of it so that allows us to capture an image that you could share with somebody else.  But the limitation of that was that it is a static image.  It’s not across the whole slide and it wasn’t easy to share. 

So, you know, there isn't the associated database or software that you could easily share the images to your colleagues in remote locations.  We then moved into tele-pathology, and to me this is when it really got interesting where we had the ability in a remote location to log into your microscope to move the stage to capture different areas within your slide and to view it on a computer monitor.  So that was a huge advancement from the previous set to tele-pathology.  It was an extremely exciting time.  The limitation of that really was the fact that it wasn’t easy to capture whole slides, and the images, when you tried to capture them and stitch them together, they were not very seamless and it wasn’t easy to have it all integrated into a digital environment that enabled image analysis. 

Then we moved on to mind scanning technology or virtual microscopes where we had to incorporate it in a way to do actual whole slides digital imaging, so now you can capture the entire slide, the software enabled you to view it easily.  You can do in and out, you have access to the entire slide very much like you would under a microscope where you can pan around and see the whole thing, and that would really be defined as virtual microscopy.  But where we are today is more than just the image.  It has the image integrate into your environment.  It’s how you interpret the slide, and it’s also how you store the slides and the associated data. 

So these huge images are great, so how do you share them, how do you analyze them, and how do you access them from unknown locations and that is really where we have moved into, anything about digital pathology and how we live and interact with these images.  So it is evolving all the time.  I am very sure that in five years’ time when I pull up this chart again, there will be another definition of where each of these steps are now considered the minimum requirement really for digital pathology.  You have to be able to digitize the whole slide.  You have to be able to share the information easily and you have to have tools that let you interact with the image.  So I'm looking forward to seeing what the next step is on this chart; very much so.

So, what is driving the adoption of digital pathology and why are people even talking about it?  We have touched on some of the benefits but there are really two things that drive the adoption of digital pathology.  There are external factors and then there are the internal factors that you are trying to solve.  If we look at the external factors, unfortunately the rate of disease is increasing globally so there is more and more glass that has to be looked at, there is more patient tissue, there is more work and more cases that have to be assessed.  And there is also a decrease in the number of pathologists, so less and less pathologists are being trained today globally.  So the number is decreasing, and also the subspecialties are decreasing. 

And all of these pathologists, of course they are in different locations.  Some of the best are scattered across the world.  And, again, going back to that idea where if you want a second opinion, if you even need a first opinion, are you going to be able to access the expert that you need.  So these are some of the reasons why digital pathology has come aligned, why it has been evolved, these are external.  Another internal condition is where, you know, when you develop a new product or a new solution or any new technology, you want it to be - - on the access is better after, or it’s cheaper than the current technologies, so if we think about microscopes as the current technology, we want in some ways for digital pathology to be better than that. 

So how do we do that?  Well, with digital pathology you think about science is becoming more evolved, the tests are becoming more complex, so for example I mentioned that today there is RNA and IHC on one slide.  How do you review that?  That is very, very difficult to score.  A possible way of digitizing and then running the image also enables these better types of assays, these more complex assays that give us more information.  We also touched on the fact that, you know, we are all humans. 

There is human error there, and we hope that using digital pathology and image analysis can help reduce your human error, even if it’s the fact that just using digital pathology to reduce transcription errors and bar code labeling, things like that that we try to reduce the steps and try to make it more automated to try and reduce the number of human error.  There is also the better consideration to workflow, so a great example of what you can do with digital pathology, you can't do the microscope.  You just very simply are reviewing multiple serial sections at the same time. 

So we all know when we look under a microscope, you look at your H&E.  If I want to look at any of the subsequent sections, I then move over to the next slide and then look at that.  And if I want to go back to HNE, I am flipping between the two.  A great advantage, and it’s very simple, but a great advantage of digital pathology, is I can open up many, many sections and view them all on my computer monitor at one time.  I can link those images and pan around the slides so I can easily see the same areas of interest across the serial sections.  So it makes it much better than just looking down at a static image under a microscope. 

There is also the access of faster, so in some instances in digital pathology it’s not faster.  For example, if you have a pathologist looking at a case of HER2, it will always be faster on a single slide under a microscope.  However, when you start to think about speed with hosting slides to different sites, that gives time to the patient.  When you think about you are looking at FISH samples, it is quite difficult to go down to the room where the microscope is whereas if you use digital pathology you can look at that in a remote location. 

Or when you start thinking about large batches of slides and not a single slide, we are now moving to many, many cases to look at in a day, you can use image analysis to try and automate that and you can use digital pathology to increase your workflow efficiently by having all of your slides collated organizing your workflow and you can sit down and review them.  And, finally, you know to a less extent, digital pathology in some ways is cheaper and that is where we really look at when we are doing toxicology, so these are broken down under running high volumes of slides, true image analysis.  There is a lot of workflow efficiencies using digital pathology which we will get to shortly.

So my particular area of interest is image analysis, and the image analysis can also be thought of as a very complex black box solution.  And really it’s not.  Image analysis is just a term that is used to describe any type of software that helps you interpret an image and that can be─ it can be as basic as counting cells, measuring margins right up to the sophisticated image analysis in trying to help detect tumor or necrotic tissue.  So that really is the term that describes anything that helps you interpret an image.  To me, image analysis is always about turning an art into a science.  You know, when you look at how pathologists look at tissue and understand the very complex structures within tissue, it really is a very, very difficult science to do. 

And to me image analysis is all about assisting them in that and trying to turn it into less of a semi-quantitative, but more into a fully quantitative.  And it really is just about enhancing their current workflows and what they do today.  So there are some benefits of image analysis.  You know it is about enabling the results to be accurate, so you always get the same results with image analysis.  It would always be the same numbers in the computer.  It would be accurate.  It would be truly quantitative, so again when I am seeing a lot of eye marker evaluations, we had scoring pathways that were zero, one plus, two plus, three plus. 

Well, what about if there were four plus and five plus, but we weren’t even considering─ what if the scale wasn’t even linear?  So what image analysis enables us to do then is to turn these semi-quantitative scales into a truly quantitative continuous scoring system that can try and find different cohorts within our patient cohort.  And that is a huge benefit to image analysis when you are trying to evaluate new biomarkers and potential biomarkers by not having to predefine co-points, by using image analysis and follow-up information to try and find the true cohorts.  And then finally again it is reproducible.  It will always give you the same results.

So I would like to show an example again thinking of image analysis as an aid to the pathologist to try and help scientists do better at their job.  We all know that there are limitations with human vision.  That is a fact and where say, for example, in this image we are looking at the orange circle in the center.  And, to me, the orange circle on the right looks like it’s larger than the orange circle on the left.  Well, realistically if I take away the blue center of the circle that are surrounding them both we see that the orange circles are actually identical in size and shape. 

It’s only the fact that the blue circles around us are just distorting our perception of the orange.  So this is a limitation with everybody’s eyesight, and really it looks interesting, but if we think about at work trying to find tumor nuclei, we will be affected or we will be influenced by the cells that are surrounding those tumor cells or if we are looking for lymphocytes, anything to do with circles and shapes, we are going to be influenced by the context and the tissue that surrounds those features we are looking for.

The second way that we are often influenced by our eyes and by our human limitations really is around color.  So we all perceive color differently.  We are all influenced by color and thresholds differently.  So, for example, in this image we see two colors of gray here.  We have got the darker gray here in A and a lighter gray here in B.  But if we transpose the A off of the B, the A onto the B, we can see that they are actually the same color.  It’s only that we are influenced by the shadow here so with our perception of the color is different when in fact they are the exact same.  And this is the biggest thing that impacts our vision. 

This affects me every day when I am looking at slides.  For example, if you are looking at HER2 it has a lovely crisp staining within the membrane, it can be quite easy to see what is a one plus, two plus, or three plus.  Well, if you are comparing that HER2 and you have got some cytoplasmic bleed or if you have got some adjacent staining in a nearby area of the tissue, your perception of the color will be influenced.  It will be influenced by the slide you looked at before and it will be influenced by the type of staining within that tissue.  And this is very well published information. 

We have seen publications that talk to the impact of scoring tissue microarrays and what order people score the TMA cores have influenced their interpretation of IHC.  So there are many, many publications on this.  I find it fascinating.  I think that we know that there are great pathologists out there that can overcome this.  They are trained, they are experts.  But if we were to use something like image analysis purely to be involved in color interpretation and counting, these are menial tasks that we could very easily train a computer to do that would eliminate these human limitations.  I think we should evaluate using those in a more routine way.

So kind of leading into that, you know I mentioned already that we have different types of molecular tests that can be visualized in fluorescence or chromogenic areas, but we could also have single and multiplex or dual plex.  And each of these types are more complex with each of the scoring.  So, for example, a great example of single plex in immunohistochemistry typically is something that your ER, your HER2─ but there are difficulties trying to create your own internal cut points for what is a threshold of a zero or a one plus, two plus, three plus, and that is very open to interpretation.  And it is very variable from person to person.  And it can be quite difficult to do, and it is influenced by pre-analytical variables, so a one plus today could be a two plus treatment depending on how the tissue is processed or how long it was fixed. 

So these are all considerations that you know does impact the single plex or the image.  We also are seeing a huge increase in the multi plex testing that is there today, so today typically when we think about multi plex we can think about things like FISH technology, and we think of HER2 and FISH as a very, very common application when we are trying to evaluate the ratio between two colors.  And that can be quite difficult to do.  It can be very time consuming to do two different colors and try and count off.  But we are starting to see more and more plexes coming in, and we have plexes out there today with four or five colors.  And it is hard to decipher between the different colors. 

And, again, it’s influenced by our human perception whereas if we use something like image analysis or digital pathology where it is a computer interpreting those colors and counting those dots, we really can try and eliminate some of the errors that are introduced when you review these things manually.  We know that with every assay and with every test you have to consider if you are going to do a fluorescence or in the chromogenic space.  And there are pros and cons of each, and I think with our chromogenic tests you know that it is very easy to do.  It’s quite inexpensive and you don’t need specialist equipment that, you know, a typical microscope can allow you to review those samples.  

The downside of it is that it is not very readily available to multi plex them because you are in the chromogenic space so there are not that many chromogens that, you know, you can layer on top.  And it is quite difficult to review if you do two plexes.  We then move into fluorescence, so fluorescence is wonderful in the space that is very specific.  It’s also very lendable to multi multiplex so you can layer on as many layers as you like in the fluorescence space.  Well, one of the limitations of fluorescence are, of course, that you have to calculate that they are more expensive to do.  You do have to have specialists and a more sophisticated microscope to visualize it. 

And it’s typically that not every lab has access to either of those two things, so it is just another layer of complexity, but what we do and certainly in my job, we see it growing, these cases growing every year.  So I really am just trying to highlight some of the difficulties here with each of them, and there are pros and cons to everything and they all have to be considered when you are designing an assay or thinking about a test that you want to develop.  So really just thinking about when you are designing these tests, you should consider image analysis that can help in many of these cases.  Certainly they help with the single and the multi plex. 

When we think about fluorescence it can be digitized and reviewed remotely.  And with chromogenic it can help really increase the throughput with the slides. So there are many pros and cons with each of these assays and applications.

If we think, if I want to summarize really the true advantage of image analysis, it is a fact that it can help remove subjectivity.  So it is truly quantitative.  It does assist in decision support.  You can have real time results when you are looking at the tissue rather than waiting on a colleague to come in.  You can use image analysis as that support.  It is used commonly for novel biomarkers, so where the cut points are not known or where it’s multi plex, you see a lot of our scientists and a lot of researchers using image analysis when they are developing new biomarkers to try and evaluate if the biomarker is a go or a no-go.  Image analysis has been used more and more in this application. 

You can't standardize results, so what we try to do with image analysis, again, is always the same.  What we’re trying to do is to remove observer variability of inter- and intra-observer variability.  And the fact that you can get truly continuous data, so again coming back to that there may be a different subset of patients or subset of staining that makes an impact, and not just using one, two, three.  You can increase efficiency, so the turnaround time, if you have large slides to review, we see a lot of it from the toxicology stains, people using immune-analysis to really increase throughput. 

It can reduce decision time where you can share a slide, do an image analysis, and that will be faster than trying to post slides anywhere.  And, again, it is in the permanent records, so it’s wonderful when you're doing biomarker evaluation.  I can look up a study that I've got a month ago, six months ago, and I can actually look at the data and look at the very area of tissue that I ran that analysis on.  And as you all probably know when you pull up a slide from a month ago, it’s not as easy to go to the exact area that you looked at.  You have to annotate the slide.  When we use image analysis it’s very easy to have that permanent record always associated with your slides.

     There are speed advantages of image analysis, and really what I wanted to highlight here is that sometimes we think about image analysis in that it’s quite cumbersome, and running it on my laptop, it takes a long time to do.  But I think I would encourage everyone to think about how technology has changed and image analysis now can be run on servers, can be run on cloud situations.  And where that really helps is increasing scalability, so you can run an image analysis locally within the software on the computer and that is where I started with my own research.  But today, you have an economy scale where you can do batch analysis across large volumes of slides that can run overnight, that can run over lunchtime, morning, or evening.  And you can walk away and come back and have the results from a large cohort of slides. 

So it really does have an advantage of speed, and I've seen in my own work where I review tissue microarrays, and once I use image analysis the throughput was greatly increased because I have the software to try and speed along the process.  There is also the advantage of using software with pattern recognition where I don’t have to manually write and annotate all of the slides.  For the reasons of interest you can use pattern recognition to identify the regions of interest, and then run the quantitative analysis.  And, again, that is where there is a huge efficiency in speed.  We are going to speak a little bit more about pattern recognition in a few more slides.

     So I'm always asked what are people using image analysis for, what is the application, what is the most common thing you see people use it for.  And I took a look at some publications that are out there today, and really I could classify that today as 51%, the majority of publications, are all in oncology.  There are some other applications, of course.  What I find particularly interesting was that the number of publications that are writing about validating image analysis is decreasing even from 10 years ago or from 15 years ago, the majority of the publications were explaining how image analysis worked, how they validated the image analysis. 

That year, that percentage, is now sitting at 12%, and really what happened is that the people aren't just adopting it in certain use cases know that it works, believe that it works, and they are just referencing prior publications.  So today it would be quite uncommon to try and see a publication that shows the validation after a nuclear algorithm on ER or PR, whereas 15 years ago that’s where the majority of the publications were setting up.  So I personally find that interesting.  I see that there is just a lot more references to prior technology, and more and more publications are using the application of using image analysis as a means to evaluate oncology markers.

If we look specifically at what are the most common applications today, it still is in immunohistochemistry.  That is where I see the most people using it.  There is a large portion of immunohistochemistry that also focuses on pattern recognition so finding the area of interest and then do your quantification.  But in immunohistochemistry, they are really still leading the charge.  Again, there are really a lot of applications out there on the core ER, PR, HER2, T67, P53, but the majority of the publications now are moving towards, again, novel biomarkers and looking at new potential assays for ALK or PD-L1 where they are using an analysis to find the thresholds or the cut points of that particular biomarker.  The second most common application today is FISH, and we do see again you know that FISH is less common of an application than IHC, but the image analysis, there are a lot more people using image analysis for FISH and a percentage are using it in routine IHC. 

And really that is the cause of FISH.  It can be quite cumbersome to count, it can be quite difficult to do, and there are a few things that image analysis lends itself to very well.  Finally, we are seeing a lot more people adopting RNA ISH technology out there.  There are a lot of publications speaking to the merits of RNA ISH, and particularly would do a plex, we see a large percentage of those publications that are using image analysis.  Again, they are referencing mostly the reproducibility and standard scoring system if you use image analysis, and the high throughput.

So, you know, often as well I am asked to classify image analysis, what is it and how do we do it.  And really you can mark them as image analysis where you are typically finding something, and once it’s - - it’s quantifying it.  So examples of these findings, there are a lot more pattern recognition software out there today.  There are a lot more people trying to come up with a technology that can find what I'm looking for.  So whether you are trying to find mitotic figures or lymphocytes or you are trying to find necrotic tissue or stromal or tumors, there are faster technologies that are out there today that can try and find the patterns. 

Today it’s all based on the pathologist and the experts, and every software application is typically handed to the pathologists so you can always argue that the software is only as good as the person training on it.  But today, the findings typically have been used as a precursor.  And the precursor to quantify, so for example, finding the tumor regions, then quantify the biomarker within those tumor regions, and the quantifying is very much all run by biomarkers, so giving the intensity scores, ratios, and areas to count the cells, so it’s pretty basic information from the quantify and very easy for a computer to generate counts, intensities, ratios, and all of the things that have become quite cumbersome for a human. 

So here is just an example, what it looks like if you were to go ahead and try and find software to identify the tumor.  And in this particular application, it’s very common with most software applications that the pathologist will annotate regions, so in this case they have validated a square within the tumor cells in red.  And they have annotated kind of the lighter blue color for stromal.  And the computer system then tries to learn, based on those annotations, so it’s so important that the annotations are done by the expert, by a pathologist, by somebody who knows what they are looking at because the computer will take that and try and learn it itself. 

So it can be machine learning or deep learning, but they are just trying to learn from the cohorts of annotations that have been drawn by an expert.  Once it has learned, it then runs across the whole section, so here you can see a higher magnification of tumors that have been identified in red.  But in the background you can see that the analysis is run across the whole slide, and the tumor regions are in red.  The ? have been identified as a darker blue and the stromal regions are then highlighting the lighter blue color.  So what does this do?  It gives me a percentage of tumor within the slide which is of course useful, but it also gives me a precursor to the quantitative analysis. 

So if you were to run image analysis to try and quantify a nuclear staining, here you don’t want it to run across the stromal regions here.  You wouldn’t want it to run in the tumor areas, so you would use the pattern recognition software to find tumors, and then run the analysis, a quantitative biomarker analysis within that tumor region.  And this is just a markup.  This just shows you that─ most of the image analysis, they will show you a markup of what the computer has classified as tumor, and also what the computer has classified as a nuclei and the intensity of that nuclei, so you use the expert to override the scores and override the markups.

The next slide just gives you a little bit of more of a visual on what those quantitative biomarker evaluations look like.  So in this case, you know, there is typically a nuclear algorithm, and a nuclear algorithm, in this case, is trying to evaluate Ki-67.  So here the markup is showing you in blue what is considered to be negatively stained, and then in red what is considered positively stained.  And, again, most software applications come as a standard protocol.  But you, the expert, can then change what the threshold is for positive or negative. 

The outputs from image analysis, again, it depends on what software you are using, but generally the outcome will be things like total nuclear counts, intensity scores, the number of nuclei and the percentage of nuclei that is within the classified of 0, one plus, two plus.  I think what is important to note as well is that all algorithms have to be trained, so in this case it’s trying to find tumor cells that are positively stained.  An image analysis is only as good as the person who is teaching it.  So do bear that in mind, and you know when you see markups like this, they have most likely been optimized and trained for that tissue.

I think one that is always of interest is really the HER2.  And HER2 can be difficult to score, particularly for the two plus cases, and really that is where image analysis comes in as a huge benefit.  But what is interesting in membrane image analysis is the fact that a computer can very easily tell you whether it is a complete membrane stain or not.  And, again, for HER2, we should be considering the completeness of that membrane stain.  So you can see in a human it might be difficult to try and assess about a complete membrane and what the intensity is.  And here you can see in these markups, you know we have intensities based on color coding, but we also get the markup to see if there is a complete membrane or not. 

So there are these added advantages again that image analysis can do so easily, but it’s a little bit more time consuming for a pathologist or for a person to actually evaluate.  Again, in this case, these are typical examples of results.  You often get your HER2 scores, so it is a zero, one, two, or three.  You often get the number of cells that were evaluated, the intensity of the membrane, so these are some examples that you know I am truncating the list here, but this is just to give you an idea of the type of metrics you can get out of the standard membrane algorithm.  And, again, it’s so important for the markup, you're the expert, you always get the markups.  This is a permanent record that I am referring to so you could open this is weeks, months, or years, and you will be able to go back and see where the analysis scored and what that score was.

And one thing to know about image analysis and the workflow through this, the workflow is very different for you in these cases.  So, for example, there are image analysis packages there that you can adopt.  There are free image analyses, there are commercial, there are many made for different types, but you should consider what your workflow is.  So if you are a post-doc who is doing an image analysis and you have a million different types of applications, you are going to want something that is pretty flexible and that has a wide toolkit that you can adapt and change, and really that you can open up and go under the hood and get deep with the product and amend it for what you need it to do. 

Whereas we have other applications, for example, more in a clinical application, you don’t want the system to be opened.  You want it to be pretty locked down and well validated, extremely easy to use, and extremely easy to integrate and get your results.  So these are things I would always encourage people to think about.  It’s not just about the algorithm, but it’s how you want to use it.  If you want to use it in a full slide and just areas within the slide, do you want to integrate it to your ILS system, do you want case reporting.  These are all so important to consider for image analysis because once you get that data, what are you going to do with it and how easy is it to get that data from the system.

So I want to just highlight as well that with image analysis through the course and the adoption curve you have to get comfortable using the software.  And that takes time at the beginning, but I think what I do highlight here is the fact that when you set up an image analysis protocol, it’s very much like an IHC protocol.  You are not going back and changing them unless something has changed in your brief.  So, for example, once I set up my dilutions for the disease state, on my auto-stainer, I am not going to go back and re-change that every day.  I am very much the same with image analysis.  When you are doing routine analysis say, for example, you are constantly reviewing - - or P67.  What you would do is you scan the slide. 

The first time you sit down you are making the protocol, so that is you are tuning the algorithm to do what you want it to do.  Again, you are the expert, and it is emulating what you are doing.  You, of course, have to validate it and make sure it is doing what you are doing, but then you can go through the process of either annotating the regions of interest or using some pattern recognition software to find the regions of interest, analyze, and review your data.  So you can see there are a number of steps there when you are setting up your protocol.  

And that is going to take time when you are starting off with image analysis.  When you are comfortable with that protocol, again, very much like a protocol in an auto-stainer, you do take out some steps, you are shortening the protocol or shortening the steps as you become more proficient with the product.  And it is something to just bear in mind.

And so I would always give you the advice if you are considering adopting an analysis, there are many, many datasets out there that you can read, many publications.  I think the things that you always want to think about is what do I need to do with this data, how am I going to manage the data, either a database associated with the information, or do I just want it to go out into itself.  How do I want to interact with this?  Do I want to have remote access to the software, or do I want it to load locally on my PC, which can be fast and great, but the difficulty is I always have to be setting up that EC and that EC data is then dedicated for image analysis. 

And also it’s always so important to think about ease of use, of course, and how can I use the product, how easy is it to learn, how easy is it to teach myself or others.  And, finally, you know I just want to call out─ think about scalability.  You know, you may start off the day thinking about I want to try image analysis, but don’t - - that you are thinking about scale.  So if you have more colleagues or more samples, it is easy to scan that image analysis.  Can it be moved to a server, can I have more than one person accessing it?

So, in conclusion, you know image analysis is consistent and reproducible, and a quantitative method of evaluating molecular pathology today.  It does enable truly accurate quantification, but there should always be considerations given to study design and how you want to optimize your image analysis.  With anything, it is only as good as the person operating it, and today combined accurate research, and the most common applications really are biomarker quantifications.  And we are seeing more and more use cases of bringing in pattern recognition into that biomarker quantification.  And, finally, if you are considering adopting, do read the publications, but do think about how you want to use it.  It’s not just about the image analysis, it’s how you want to interact, how you use the data and access that margin.

QUESTIONS

Why is image analysis not used more frequently in the clinical setting?  What are the barriers?

DR. CONWAY:  So that's a great question, and I suppose the first barrier for the adoption of image analysis is the adoption of visual pathology, so not every hospital has adopted digital pathology yet.  And for many, we have regulatory costs, time, and understanding.  We see globally that there are different territories that are adopting more readily than others, and with image analysis, you know it really is a factor of what does that hospital do, is it large enough, does it do FISH, does it do immunohistochemistry?  So there are countries that have adopted it much higher than others, and for many, many different reasons.  What we are seeing is the adoption of image analysis growing considerably, so I think it is the start of the adoption curve really for this application and it is growing.

The next question is what are your thoughts on placing image analysis into the IHC workflow.  For example, the on-slide control after staining is reviewed by image analysis.  And if the quality parameters are met, the slide has passed the pathologist objective rather than subjective quality control of IHC.

DR. CONWAY:  Yes, I go for that idea. I'm very in favor of that and I've spoken to a number of pathologists that are asking for that.  So I can certainly see that being a product, and I certainly see the use case and I think it’s a great idea.  And I know even internally we've used image analysis just to test new products in our advanced staining portfolio.  So we even use it, but I certainly see that as being a product in the future.

Okay, we have time for maybe one or two more questions.  The next is I am interested in quantifying IHC nuclear and cytoplasmic stains separately.  I wanted to know which program is more appropriate to use on my personal computer.

DR. CONWAY:  So I don't know if you want like a recommendation of a person or a company’s product, but we do, for example, we have nuclear algorithms instead of─ in your case it’s probably better to use if it was - - pathology besides a - - algorithm because it would give you a nuclear evaluation and a co-localization of evaluation and you can run that locally on your computer.  So if you want to contact me afterwards, I can show you how that works, so it’s not a problem.

And our last question, we may be able to squeeze in two more, what are the advantages of using server side image analysis?

DR. CONWAY:  Yes, so for me, server slide, I mean again I use a lot of different technologies.  What I looked at the server slide was it didn't─ if I ran an image analysis on my local computer, I couldn’t do anything else on that computer.  It was working away on image analysis so it was not free to do anything else.  When you run it on a server, the server is more robust.  You can send the slide of the batch analysis and continue working on your own computer. 

It’s also more scalable so you can increase your hard drive and your space on your server so you should get more slides.  You can do a lot more on the server.  And the final thing is the remote access.  You need─ I like to be able to access my image analysis from home or my desk or a separate office or a different location.  And having it on a server is much more beneficial because you can remote in and see the progress of the image analysis and you don’t have to be tied just to your desktop computer.