The Cutting-Edge Technology of AI Prostate Cancer Diagnosis in the Era of Digital Pathology

Hi. Thanks for the invitation today. My name is Sun Kim, and I'm a founder and CEO of Deep Bio, the startup company using deep learning technologies for diagnosing cancers in tissues.

Today I'd like to talk about the cutting edge technology of AI prostate cancer diagnosis for digital pathology. In this talk, I'd like to introduce the unmet needs of the pathology area, especially digitization and combining together with AI technologies to help pathologists for generating better diagnosis of prostates and the supporting technologies.

Revolution of Artificial Intelligence

As we all know about the revolution of artificial intelligence, we need to have a lot of data sets, especially use that data set for training purpose. For training those big data sets, we need strong computing powers. Recently, due to the collection of big data and the hardware equipments advance, we now can generate decent classification models compared to humans' cognition level, which is called the deep learning technologies, one of the artificial intelligence areas. It is known that this deep learning technology can distinguish the patterns as good as humans' cognition error rate, around 2%.

Deep Learning Algorithm and Increased Utilization of AI in Healthcare Industry

There has been lots of machine learning technologies coming recently using neural networks, but those technologies provide the classification algorithms by human beings. Usually the researchers classify the feature extractions for better classification of the outcomes. Somehow deep learning technologies, AI generated their feature sets by themselves without having humans involvement at all. It turned out that kind of structure guaranteed a better-trained result and is the first algorithm to generate the humans' pattern recognition rate in history.

Problems with Traditional Pathology

As the advance of the artificial intelligence technologies coming, there have been problems from the traditional pathology area. They have used microscope devices for more than 400 years without having any digitization equipment. Pathologists are measuring cancer area; the level of the severeness of the cancer using microscope by seeing the stained glass slides of the biopsy tissue samples. Because of those characteristics, it's not easy for pathologists to write the proper report. Some of the inter-pathologist disagreement is because they are using their bare eyes, and then doing guesstimation of the pattern recognition. Also there are shortage of the pathologists globally. Annually, it is a big problem because there has been more requests of the diagnosis of the cancer from the tissues.

Answering the Unment Needs of Pathology with AI

Answering these unmet needs of the pathologist using deep learning technologies, still there has been some problems to solve. The first problem is we need to scan those big images of glass slides using the multiplications. Usually doctors are using 40x, which is 400 times of original images using microscope. Therefore, to save those images, it's huge image resolutions compared to the X-ray-based images.

X-ray-based images is thousand by thousand size, and if you save this image, then it's two or three megabytes. For big images of glass slides, and then the resolution is like 150,000 by 100,000. The size of the one scanned images is two to seven GBs, which is huge enough. You need to have proper engineering technologies to deal with those big data sets, especially when you are training, you have to manage those big images to train well. Humans are using glass slides and there are tiny area of the cancer cells, it's not that easy to guesstimate all those exact location of the cancers again using the microscope. Therefore, there are so many scattered cancerous area here and there, and then later when you find a different severeness of the cancer patterns, still it's tough to measure the exact percentage of the proportion of different patterns over tumor areas as a human being. Because of AI based technologies, we can provide time-saving through faster and more accurate diagnosis using AI. We are providing consistent results because anytime it is using the same mathematical models, and also generate better quantitative metrics for diagnosis and further prognosis estimation too.

Digital Pathology Process

The digital pathology process is using equipment to scan those glass slides and save it into the glass scanned images. Then AI gets those inputs and generates reporting, and later that can be used for further treatments.

AI Software for Prostate Cancer Diagnostic Support

In this section, I will briefly introduce the product of the Deep Bio prostate cancer diagnostic system, Deep DX Prostate.

The prostate cancer diagnosis is using Gleason scoring system. Pathologists are finding the tumor cells, and later when they find the tumor area, they provide the different severeness of the cancers, which is called the Gleason patterns. There are three different patterns based on the severeness of the prostate cancer cells, pattern three, four, and five. It's not easy to measure all those pattern areas using guesstimation of bare eyes using a microscope.

Our AI software can provide the exact segmentation information of each of the different Gleason patterns, three, four, and five. We are providing pixel-level accurate information, especially glands of all those prostate cancer cells can be found. We are providing exact proportion over tumor areas, which are needed to write the Gleason report for needle biopsy cores of the pathologists. Those are important metrics because later, urologists are using those metrics for further treatment and prognosis estimation.

As you can see in this screenshot, we are marking different colors for different patterns, three, four, and five. Doctors can see different patterns of the cancerous area. If you click those buttons, then you can see the original scanned images too.

Our clinical validated results show that the sensitivity is around 99% and specificity is 97%, and we are fast to generate those reports within a minute, around 30 seconds on average. This is useful for helping pathologists have a more accurate diagnosis of the prostate cancer.

Now you can see the live demonstration of our software products. If you upload those scanned images of the prostate needle biopsy core scanned images and click this AI button, within a minute our backend GPU servers are going through all those images and start to find all those glands-level of the cancerous area, as you can see. This is real-time diagnosis. If you click those pattern three, four, and five buttons, then you can see the original images too. You can go to the focus of the point of interest area too. If you increase those resolution later on by human pathologists, they can adjust those directions of the images for better diagnosis for their convenience, and also generate those proper reports. You can see all those metrics here, the tumor length, tumor area, also the Gleason patterns proportion, and Gleason score. Doctors can measure those cancerous area and adjust some of the cancerous area by their determination, and later report them.

Use Cases

Our products can be used for a variety of use cases. The first thing is using screen purpose. Our products can tell whether those scanned images of the needle biopsy of prostate cores of scanned images has a cancerous area or not. Second wise, doctors can use our software for helping their diagnosis using our software as the clinical decision-supporting systems. Third one will be the quality control purpose. Doctors can diagnose those prostate cancer patients first, and later they can refer our diagnosis as the second opinion, which will help generate better reports all the time. The final one will be The research and development purpose, especially doctors are doing research with our DeepDX prostate cancer right now in multiple United States medical centers.

End-User Testimonials

Our DeepDX prostate software already deployed in the United States region, and those pathologists are using our software as the quality control purpose in our CLIA laboratories. And some of the pathologists just do their testimonials in multiple magazines.

One pathologist just mentioned that with a simple click of a button, sometimes small area of concern was revealed that he would have missed. Our software can found some of the small focus area of the cancerous region, so useful for the quality control purpose. There are multiple magazines and one of the magazines, especially for CLIA laboratory employees, the Laboratory Economics in May 2021 issue, Dr. Kahini mentioned that this tool is very useful for quality assurance because sometimes pathologists re-look at some areas that they did not take originally, but AI algorithms can mark it, and that can be very useful, even though it's small fraction of the area of interest with a small cell of cancers.

Differentiated Deep Learning Technology For Better Performance

I'll talk about some technologies behind those AI-based software tools. As I mentioned earlier, those deep learning technology is one of the machine learning technologies in artificial intelligence category. Those machine learning technologies are known that researchers are providing feature extractions for classifying the outcomes and then try to enhance those algorithms to have a better determination of the feature sets. Whereas deep learnings are automatically generating those feature extractions by AI-trained models without human's intervention. We don't know why deep learning is working so well, but it turned out that deep learning's classification capability is as good as human beings'.

Deep Bio's deep learning technologies are inspired by actual pathologist activities and behaviors to diagnose the prostate cancers. For example, when they are diagnosing prostate cancers, they are seeing pattern three and four with a more smaller angles and sparsely. Whereas Gleason pattern five, reversely. We developed our internal segmentation network of the deep neural network based on those concepts, and we were able to generate the decent trained model and now we also developed technologies for intelligence annotations, because even though you have a lot of datasets, scanned images of cancerous slides and regions, still for generating refined model of a segmentation network, then you should have the area annotation for those slides, which pathologists are never doing it in their actual clinics.

Therefore it is time-consuming for them to generate those annotated datasets. Deep Bio's technologies enhance those intelligence annotation technologies to help pathologists to generate the proper segmentation annotation. We have our continuous improvement technologies behind on our neural network backbone, so that whenever you have a more datasets, it gets improved gradually. As you can see on those image sets on your left corner, when you are generating Gleason patterns of scanned images, then usually AI are using small chunk of tile and then try to find the cancerous areas later on. And based on those classifications, they are generating heat map for better future extractions and defining the tumor area.

Depending on each of the hospitals based on their combination of chemical staining modals, H&E, or the age of those chemicals, or the thickness of tissue cuts, sometimes the color variation are pretty much severe between multiple medical centers. If there are multiple scanners to scan the same slide images, it is possible to generate a different color segmentation, color tone and because we don't know what kind of features deep learning technologies are using, there is a possibility of the degrading performance because of that variety. Our technologies can always sustain the same level of accuracy based on technologies behind that.

Using the same glass slides, if different scanners scan those, you can see the color tones are quite different. Leica AT2 and Aperio GT450, there are differences of color patterns here. Even with the same scanner brand, if we are using different model, GT450 and AT2, you can see there are slight differences of those color tones. In your viewer, ImageScope has a color profile. If you change those color profile, then also it change the color representation, tone, and manner of them, which might generate degrading of AI performance.

We just collaborated with nine different hospitals in South Korea, and we found that some of the clinic's diagnosis which was handled toward the urologist, especially there are real clinic's diagnosis of prostate cancer, those were benign, which is no cancer there, but we found some of the areas has a cancerous region. To have a clarification here, we just do the cytokeratin 14 stain, and if there's no beige cell without having any brown color tone border line of gland, then it is a cancerous area. You can see those images here and these H&E stained images, we marked different color, orange-ish or bluish color here. And if you see the corresponding CK14 stain results right next to that, there's no brown cell happening here. Therefore, it can be said that our diagnosis can be useful for quality control as a second opinion too, to have a better quality of life for the patient.

Clinical Validation

Deep Bio's DeepDX prostate cancer diagnosis system did a clinical validation, and we published our clinical validation paper in 2019 in Cancers. In this journal, we introduced our performance by comparing the gold standard of three pathologists, and then we compare our results with gold standard sets. And each of the individual pathologist's performance is also compared with the gold standard to find out the interobserver variability among pathologists. It is well known that there are lots of disparity between different pathologists on diagnose spatial cancerous area and severeness of cancer because they are using bare eyes and microscope. There's also interobserver variability. Based on that, we just confirmed that our AI-based diagnosis systems did as well as the human pathologist, the gold standard. Uou can see those confidence metrics here.

Total number 1,833 H&E stained glass slides of prostate needle biopsy cores were collected from two different hospitals in South Korea. After the normalization of the data, the slides were digitized using AT2 scanners at 40x multiplications. This is the resolution of 0.25 micrometer per pixel. After digitization, 700 images were selected for the validation sets and balanced according to the grade groups reported in the original diagnosis. The remaining 1,133 cases were used for the discovery sets.

The diagnosis results were compared with reference standards, and you can see the Kappa score of deep learning algorithms with a reference standard. You can see high diagnostic concordance was shown between DeepDX and the reference standard compared to the original hospital diagnosis. It's a 0.907 quadratic weighted Cohen's Kappa coefficients, which is a good result overall. DeepDX can be regarded as the same level of the human pathologist diagnosis ability.

Deep Bio's recent research results published in Nature Partners journal Digital Medicine, July 15th, 2021 and then the title is "Yet Another Automated Gleason Grading System by Weakly Supervised Deep Learning." When you have lots of data sets of scanned images of prostate cancer needle biopsies, there are multiple approaches of doing the experiments. Maybe you can use the outcome, like a Gleason score, the numbers with a given image, and train them based on that.

For generating the segmentation model, you may have annotated data sets, especially for different Gleason patterns, 3, 4, and 5. The pathologists need to annotate those segmentations, and later they can be used as the trend inputs. It's time-consuming, and usually in the real clinics, pathologists never annotate any segmentation. They only use the microscope devices and use their bare eyes and look through all those tissues, and later they determine what is the final Gleason score based on the Gleason patterns recognition by themselves.

In this paper, we presented a novel supervised deep learning based technologies trained only from the original hospital's diagnosis slide-level annotations. The total 7,618 stained glass slides is containing a single prostate in the biopsy core, and their respective diagnoses were collected from two different hospitals. After analyzing all data sets using the AT2 scanner, we scanned them at 40x multiplications. After the digitization, 6,664 slides were selected for the validation sets, and 936 slides were used for the discovery sets.

When you need to train the segmentation model, you need to have a segmented input area annotation. If you want to generate the Gleason grade, which is the sum of two numbers, Gleason pattern 3, 4, and 5, the largest pattern first and second largest pattern second. You need to have a lot of slides, because when you have slides and corresponding Gleason grade, which is two numbers, then the total number of images are the number of slides. When you have 6,000 slides, then you only have 6,000 inputs, which is less number of training sets, especially for applying deep learning technologies. Because of that, to generate the Gleason grade group, you need to have some of the area annotation and each of the images just trained using that, and later each of the image's patches can be diagnosed as the different patterns. Later in the slides level, you can generate the Gleason grade group based on that.

Our novel approaches can generate similar grade group without having whole annotations; only by having hospital level of the diagnosis, the two numbers, and given slides with a less number of slides, we can generate pretty good Gleason grade group.

As you can see here, the result shows that ROC-AUC is 0.938, and sensitivity for cancer detection is 93.6%, and specificity is 96%. The grade group prediction, the quarterly kappa is 0.897, which is decent valuation here.

From needle biopsy to TURP, we keep training our model for helping pathologists apply this on different modality of the tissues. In USCAP 2020, we decided to apply some deep learning trend model using needle biopsy cores for diagnosing transurethral resection of prostate images, TURP. When a patient has prostate hyperplasia sometimes they need to do this type of surgery, which is called a transurethral resection of prostate to help prostate cancer patients.

Doctors always need to go through all those scattered images, where there might be a possibility of cancerous lesions. It is a tedious and time-consuming job for pathologists, and we keep targeting those tissues to find the cancerous area. In total, nine malignant and 160 benign whole slide images were included in that study. Of 90 malignant whole slide images, 98 contained prostate adenocarcinoma and one contained urothelial carcinoma. There were no cases of disagreement between the hospital's original diagnosis and the pathologist's review.

After applying this convolution neural network model trained using prostate needle biopsies, the false positive rate was high because some of the tissue's artifacts that were not observed in biopsy samples were in TURP specimens. This result seems interesting, and it seems like the performance wasn't that bad, but it's not the best cut.

In USCAP 2021, we keep applying similar idea to have better diagnosis results. In this study, we performed a fine-tuning of neural network using benign TURP and transurethral resection of bladder whole slide images to advance our model's performance. As you can see, the performance level in this confusion matrix was pretty good. Accuracy number was 0.943. Incidental prostate cancer is reported around 4% to 16% in TURP cases. In the very large tissue area of TURP, the proportion of the cancerous lesion area is very small because that small cancerous area may be missed during microscopic diagnosis. Therefore if we can find out pretty small area of the cancers using AI, that can be helpful for pathologist diagnosis. As I mentioned, the previous study, the researchers reported on automated diagnosis of TURP whole slide images by traditionally applying a convolution neural network, which was developed for the diagnosis of prostate needle biopsies. In this consecutive study, the researchers performed a fine-tuning of DNN using benign TURP and transurethral resection of bladder whole slide images to advance the model's performances.

3,898 H&E stained glass slides of TURP and 22 H&E slides of TURP were collected from Seoul National University Hospital, and all were scanned using AT2 digital scanner. After digitization, 3,682 TURP whole slide images were selected for the validation sets. For the discovery set, 2,160 TURP whole slide images, 22 TURB whole slide images were used.

In this study, the researchers focused on the weakness of the previous model, which showed false positive detections for cauterization and squeezing artifacts of tissue periphery. Adding benign tissues in training, the performance of the model was significantly improved compared to the previous model. The algorithm's detection accuracy, sensitivity, and specificity were 94.3%, 94.5%, and 94.3%, respectively.

We did multiple clinical validations, also user test, using our deep learning algorithms or without using our deep learning algorithms. We have multiple metrics. Here, one example is the time spent in diagnosis without deep learning aids, with deep learning aid. You can see that using our diagnosis in advance of the doctor's diagnosis, they can shrink the total time of the diagnosis quite a lot impressively. That will generate much better performances with multiple patient slides, and that is productive, a daily basis of workflow of the pathologist in an hospitals.

Providing Ways to Create Value for Medical Staffs and Patients in Cancer Medicine

We mentioned the medical needs between digital pathology and AI area to help a pathologist, and we provided our AI-based software system for helping pathologists to have productive daily outcomes in workflow, and also introduced multiple technologies behind our software and some research results. The final comments will be how deep learning and AI can be toward the precision medicine in different phases of the product roadmap.

As I already mentioned, those deep learning technologies are mature technologies. They are good for classifying different patterns of the images, given the similar segmentation annotation inputs with a lot of data sets. Then the trend model can distinguish different patterns very well compared to the human beings, as good as human pathologists. We just apply this for the clinical validation and also for helping pathologists to diagnosis of different modality of the cancers. We keep generating more number of products line for diagnosis area. One of the good things for applying AI in diagnosis, especially prostate cancer needle biopsy diagnosis or the other specimen, it's more consistent compared to the human beings' results. Each of the human beings, they have an inter-observer disparity. It's good to have our type of AI-based software for a second generated result.

Another strong advantage of using AI for diagnosis is we are providing more accurate metrics, the cancer severities and segmented area, and the proportion for each different patterns over tumor areas. Because of that, metrics are more consistent and more accurate compared to the human pathologist.

It is good to apply those numbers for the prognosis estimation. Based on those diagnosis result of AI, also AI can lead better prognosis estimation model to apply those metrics with given clinic data sets. The final stage will be the treatment. Lots of AI-based companies are trying to find the digital biomarkers or some of the biomarkers for helping doctors to treat the patients properly, or some of the user stratification for helping better drug responsiveness to help patient quality of life better.

It aims to contribute to medical society through AI by enabling faster and better treatment decision-making for medical staff and patients with intelligent medical data analysis software. Thanks for your attention so far. Now, if you have any questions, please let me know. I'll be here for answering your questions.