Pathologists View, We Listen - A Thorough Methodology to Measure Image Quality During Product Development
We know how important it is for Pathologists to clearly see morphologic details in order to make an assessment of a case. That’s why we relentlessly pursue testing next generation scanner images with the aim of achieving excellent image quality. To do this, we follow a disciplined process to measure image quality through the eyes of the Pathologist.
How do we prepare for an Image Quality Test Session?
For image quality measurement sessions, we scan H&E and IHC-stained slides across many different organs from the customer’s pathology lab on the new scanner in development, and we make sure of the following regarding samples tested:
- Samples scanned include various morphology features.
- Samples include key bio markers including ER, PR, Her2, Ki 67 and p53 at a minimum.
- A multi-headed microscope i n the l ab s o Pathologists can compare digital images to the glass slides under the micros cope.
- All sessions are video recorded so we can study the test sessions and share the results with our extended optics team at our imaging headquarters .
What Image Quality Metrics do we Measure?
When measuring the Image Quality of a scanner in development, we visit Pathology l abs around the world and test digital pathology scanner prototypes with many Pathologists . We prepare testing scenarios that the Pathologists will experience in their day-to-day work life. During i mage quality learning cycles , we test, measure and record the following:
- Image quality scores of 1-10, with 10 being best (median, average and combined % of images that scored 9-10)
- Data segmented by type of tissue, organ type and bio marker
- Quadrant scoring where quadrant 1 is poor, 2 is moderate, 3 is good and 4 is excellent
- Which magnifications are used by Pathologists to review cell morphology and make case determinations
- Direct feedback from Pathologist about image color, contrast, resolution and viewing experience versus the micros cope
- Comparison scores (A/B tests ) for images from new product in development versus previous generation.
Design, Test with User, Iterate....Repeat Process
Once we have measured the image quality during a learning session with multiple Pathologists , we take those scores and all the rich learnings back to our headquarters in Vista, California and study the results in an Obeya Room (Obeya means “large room” or “war room”). In this room, we carefully study the user data as a team, including engineers , software developers, and product managers. We then identify which changes are needed to the product before going out for the next learning and testing session with Pathologists. The changes to the product are made and then we repeat the testing process.
In order to release a product to our valued customers , Leica Biosystems is committed to achieving image quality scores of 9-10 (or >3.75 i fusing quadrant-based scoring criteria) before the product’s design is completed. Often times, this may require many rounds of i mage quality testing across many Pathologists and the as sociated design changes before a product is ready to launch. This thorough process is very important as it ensures that we measure image quality through the eyes of the Pathologist.
For the use of any product, the applicable product documentation, including information guides, inserts and operation manuals should be consulted.
Copyright © 2022 Leica Biosystems division of Leica Microsystems, Inc. and its Leica Biosystems affiliates. All rights reserved. LEICA and the Leica Logo are registered trademarks of Leica Microsystems IR GmbH.