Full Version -
Pathologists are not just diagnosticians. They are increasingly guiding very important yet dangerous and expensive therapy. Unfortunately, pathology reports may contain errors. This presentation will review the challenges of creating a system to assess the quality and consistency of diagnoses produced by individual pathologists and discuss setting goals to improve the processes in anatomic pathology for enhanced patient safety. All of these aim at improving diagnostic accuracy and consequently better patient outcomes.
- Review the challenges of creating a system to assess the quality and consistency of diagnoses produced by individual pathologists.
- Discuss setting goals to improve the processes in anatomic pathology for enhanced patient safety.
- Review how improved diagnostic accuracy contributes to better patient outcomes.
Dr. Magliocco received his MD at the University of Alberta in Edmonton, Canada. He completed an FCAP Fellowship in Pathology and was an FRCPC Fellow in Anatomical Pathology at the Royal College of Physicians and Surgeons in Canada, after which he completed a postdoctoral fellowship in Molecular Surgical Pathology at Fox Chase Cancer Center in the Philadelphia area of Pennsylvania. Before joining the Moffitt Cancer Center, Dr. Magliocco was the Medical and Scientific Director of the Canadian Breast Cancer Foundation/ Alberta Cancer Research Institute Biorepository in Calgary.
In his role as Scientific Director for Moffitt Cancer Center Tissue Core, he was involved in developing new quality metrics, oversaw implementation of the successful Pilot Tissue Access program, and obtained key grants from TCGA and various industry collaborators including Biotheranostics. He also led the core to successful CAP certification in 2015 and successful review on the 2017 CCSG site visit.
He has over 150 peer-reviewed publications, and is an invited lecturer at national and international conferences.
Anatomic Pathology's Quality Journey: Aligning Administration, Lab Leadership and Pathologists to Raise the Quality Bar
Today I've selected a topic that I think all anatomic pathologists and people who work in anatomic pathology are interested in, and that's quality. We're living in a world of changing quality demands and I'm going to highlight some of the latest thoughts and advances in quality and application of quality to pathology. The presentation was prepared by me and the views are mine.
The objective of today's discussion is to review the challenges of creating a system to assess quality and consistency of diagnoses produced by individual pathologists. We will discuss how to set goals to improve processes with the ultimate target of improvement of patient safety and how improved diagnostic accuracy also contributes to better outcomes.
We live in a changing time, particularly in the U.S., moving from fee-for-service practice to thinking more about the value of the money spent on healthcare. The discussion is that payors will be moving towards a value and a quality type of payment system. Pathologists need to be prepared for changes.
Patients are becoming more empowered, because they are becoming more responsible for their own payments, they are becoming more aware of information through the internet and social networks, and are becoming less tolerant of error. As they take ownership of their healthcare, they are less tolerant of system errors. These things create a confluence of events for anatomic pathology.
Remember that anatomic pathologists are not just microscopists or diagnosticians. They are fully-trained physician, medical consultants, and part of the care team. With the changes in therapy, pathologists are becoming even more important as many ancillary tests are now essential for the guidance of toxic and expensive therapies, particularly in the cancer arena. Many cancer drugs are linked to companion diagnostics and there is currently an explosion of trials and immunotherapeutic treatments. Many treatments, such as CAR-T cell therapy, may cost as much as a million dollars or more per patient per year. Much of this treatment hinges on the pathologist's ability to give an accurate diagnosis and provide accurate ancillary testing.
It is problematic that many reviews of pathology reports have shown errors frequently occur. Depending on the denominator in some high-risk specialties and esoteric areas, error rates can be as high as 44% of the time. Up to 10% of these errors may be serious enough to affect treatment. For example, a staging error could affect whether patient qualifies for additional chemotherapy or radiotherapy. An error in omission of a companion diagnostic may also preclude a patient receiving a potentially life-improving therapy. This is an important area that pathologists need to be aware of and an area of increasing concern regarding the quality of pathology practice.
In a 1999 study, Johns Hopkins University found 86 of 6,000 pathology reports or 1.4% contained errors serious enough to change the diagnosis and could be considered critical errors.
Where do errors come from?
In breaking down the pathology, there are three critical areas of the process.
The first is preanalytics, which is the preparation of the specimen prior to receiving it in the lab and an area fraught with errors in the sense there is opportunity to mix up specimens. If an error occurs at this stage, it is hard to detect and may require advanced technology, such as DNA typing, to ensure the patient is matched to the correct specimen. Oncologists and physicians might be upset with stringent laboratory acceptance practices, but it is important to enforce the practice of properly labeling specimens prior to submission to the lab. These stringent acceptance requirements are really for patient safety.
In the preanalytical condition, management of the patient specimen is also very important. This involves warm ischemia and cold ischemia, which the duration of time from when a tumor or tissue has been devascularized and then transported to the pathology lab, prior to the time it goes into fixative. This is a critical time for living specimens because as soon as they are devascularized, they undergo physiological processes such as hypoxia and consummation of the glucose in the specimen, leading to rapid changes in biomarker quality and status. Prolonged intervals of warm ischemia can adversely affect the performance of critical biomarker analyses, so these intervals should be documented and as short as possible.
The next area is the analytic area, which is the machines and the testing process. The analytics are probably best controlled in a pathology laboratory, as we are aware of this process and it is under the control of the laboratory in terms of reagents systems and machines we use. One important thought is that standard operation procedures must be in place, must be followed, and must be enforced. It is problematic if FDA assays are pulled apart, tampered with or changed, so one must be certain that the analytical procedure is consistent with safeguards in place to look at batch-to-batch variation and to document any unexpected occurrences during analytical process.
The next area where errors can occur is in the post-analytical area, where a lot of scrutiny hasn't been given. Is the pathologist is interpreting specimens correctly? Are they trained to interpret the specimen? Is there an error in diagnostic judgment?
Pathology practice hasn't changed a lot over the years. Pathologists are medical specialists focused on handling glass slides. Information systems are not the most efficient or tailored to anatomic pathology practice. One of my frustrations in my career of anatomic pathology is that the systems I used were designed for clinical pathology practice with anatomic pathology modules added on, making them cumbersome and unsuitable for routine anatomic pathology practice.
Anatomic pathologists are overloaded with unlimited specimens to be seen per day and unrestricted intervals such as clinic times or opening periods. The field is vulnerable to abuse through overuse or increased volume. It is important they practice in a safe manner, managing only what they are comfortable with. They should speak out if they are overloaded or if they cannot make an accurate diagnosis due to the quality of case material. In some instances of diagnostic errors, pathologists claimed they were overloaded with too much case material. However, in litigation this did not stand, as it was determined a pathologist is responsible for practicing in a safe manner. Pathologists should be confident about their own practice ability and ability to manage case loads. Reputation is everything in anatomic pathology. That is all they have, so it is important that their reputation is maintained and built through a focus on quality practice.
How are we doing in diagnostic pathology?
According to a CAP/ADASP summary of 122 combined studies of 2014, 89 surgical pathology studies, 20 cytology studies, and 13 covering both, the discrepancy rates were analyzed. There was a median of 28% across all studies, with the major discrepancy rate being 4.8%. This was 6.3% for major discrepancy in surgical pathology and 4.5% for cytology. Major discrepancies impact patient care in terms of change of diagnosis and so on. These numbers are concerning because every 100 patients adding up to 6.3% may have an error that influences treatment.
At the Moffitt Cancer Center, the standard is to review every external case that comes to the center for review of the pathology and for reassessment by tumor boards prior to new therapy initiation. Internal versus external review data shows the internal review data is lower at 1.3% versus 7.4% discrepancy rate for external reviews. A higher external discrepancy rate is often seen in major academic centers because they have an internal quality control process that removes major discrepancies before issuing final reports.
What processes are put in place at the Moffitt Cancer Center?
I will share these with you so you may consider them for your own practice. We have 26 pathologists and are increasing our practice, adding 2 or 3 every year. The first thing is a focus practice review of every new pathology. At our center we have a subspecialty practice so we tend to restrict the pathology practice to one, two or three areas of practice. For example, a breast pathologist may cover GYN, as well as cytopathology, or a neuropathologist may also cover the autopsy service.
Prior to initiation, a plan is set out. The section head will identify the area of practice at an interval of time, generally 6-months. After that time, that pathologist's work is reviewed internally and the external work is reviewed as well. In our practice, cases are often sent out to other centers for second opinions. We collect those and monitor them, as they can be a check on our own proficiency in comparison to diagnostics at those other centers. If that goes well, that pathologist is cleared for routine work.
Second Expert Review
The other thing we do is second expert review of all new Moffitt patients. We see 55,000 unique patients each year, and about half of those patients come with diagnoses from other centers. It is our practice to review the pathology of all outside material prior to initiation to therapy at Moffitt. Part of the process is all new patients are presented at Tumor Board Rounds at Moffitt, which is very busy. We have 13-14 tumor boards that occur every week, so there are 1-3 tumor boards every day at Moffitt. They're a joint production of clinical divisions and pathology subspecialty experts working together to present cases. Typically, at Tumor Board, up to 30 cases may be reviewed over a 2-hour interval.
Moffitt's Quality Department has identified policies regarding consultations and managing errors. If there is a significant error, generally a second opinion is obtained within the department. We also have a policy of notifying the care physicians that we have identified that error. In addition, if the error involves an external pathology, our policy is to notify that external pathologist out of courtesy that there was a difference of opinion. We have also mandated synoptic reporting to standardize discrete element reporting, using CAP Synoptics, which is one of our quality indices whether synoptics are being used correctly.
Daily microscopy rounds
Daily microscopy rounds are also valuable. At 1 p.m. the pathologists can attend the multi-headed microscope conference to bring case materials to fellows and trainees. We have 12 residents and 7 fellowship programs. We have social events that give pathologists the chance to share and discuss challenging cases and set their thresholds for atypia and other aspects. There are great learning events as well. We also have a process for formalized interdepartmental consultations where hard copies are issued so pathologists can write out their opinions to be retained for future reference. We have a mandatory quality assurance meeting monthly for pathologists. If pathologists do not meet minimum attendance requirements, they may not be eligible for promotions or bonuses. Topics such as policy and quality are discussed and key performance indices are reviewed. Managers and departmental administrators may also attend these meetings.
Key performance indicators
Key performance indicators are essential to know if you are on track. The key performance indictors must be meaningful and relatively easy to collect, such as turnaround time on a frozen section or number of cases going out to an external center. These must be reviewed. Targets must be identified and achievable. If there is deviation from the target, it must be determined if the target is appropriate and if the deviation is an indicator of quality risk, corrective action must be taken.
Monthly morbidity rounds
Monthly morbidity rounds have also been instituted in our department, where errors or near misses are discussed. Pathologists involved are masked and the cases discussion is for learning and improvement of safety.
Our department also created a Safety Committee, which works independently of the administration, and is used frequently when concerns arise. Members do not hold formal positions in the department, but review incidents and make recommendations for improving processes within the department. It is also policy that every critical incident undergo review as sentinel events to ensure we learn from mistakes and they don't happen again. Every mistake is an opportunity to improve processes and ensure quality improvement.
External proficiency testing
External proficiency testing and reviews are powerful. They are great ways to ensure that practice standards are in par with those of other centers. They are also useful in removing personality from the review process because it can be a challenge to review the work of your colleague whom you work closely. It is more comfortable when the work goes out to an external center or to identify the experts elsewhere so their work can be looked at. Any issues with quality can be addressed without discomfort.
One tool used at Moffitt is incentive and bonus plans. At the beginning of the year certain mandatories are in places, such as the quality assurance meeting attendance requirements. Targets are sent in terms of turnaround time, quality reporting, etc. Bonuses and merits are tied to these as part of an incentive plan to improve overall quality.
Review of pathology by experts
Review of pathology by experts is central to high-quality practice, particularly in the complex area of cancer care, and is an important area to focus on. Moffitt recommends as a safety measure that all cancer patients confirm a malignant diagnosis by expert second, which would ideally occur before commencement of definitive treatment. We decided to create a subspecialty practice at Moffitt. We have 26 anatomic pathologists and 13 hematopathologists that are ranged into groups anywhere from 1 to 5 subspecialty experts. This is essential to establishing deep expertise in an area. Cancer is a rapidly-changing field in terms of challenges and trials, so a subspecialty practice is the way to keep up on that. It enables a team-oriented approach and creates bonds between the oncology team and pathologists so trust can be established. One problem is it is not as efficient general practice in distributing work equitably across teams.
What happens in a subspecialty practice is that it can be difficult to justify to hospital administrators the need for additional subspecialty experts in areas of low volume. The teams may view their work as being different, so work volume between different teams is hard to measure. For example, cutaneous pathology teams generally general more RVUs per unit time than breast cancer teams, creating issues in understanding equity in distribution of work. The complexities of subspecialty practice have not been adjusted over time, so that level 3 surgicals tend to be overrated in the WRVU system, whereas high-level, complex cases are undervalued.
In a complex cancer center, subspecialty experts do significant work not captured by the RVU system, such as preparing for Tumor Board rounds, doing internal quality programs, and teaching. Significant quality work is not being properly captured by current workload systems, which is a problem. Other systems out there such as the U.K. and Canadian systems are generally based on time measurements and anatomic pathologists need to look more carefully at how work is measured and valued so that they get compensation and appropriate time needed to maintain patient quality. Some of these challenges are particularly acute in subspecialty practice centers where all cases are complex and critical to trials and experimental procedures, so caution is very important in these areas.
Synoptic reporting is useful in that it ensures completeness of reports. It enables pathologists to remember to report all critical elements completely, but it should be cautioned that a generated report may look accurate, but not be accurate if incorrect data are entered. I have yet to find a good informatics tool that facilitates synoptics generation conveniently in terms of providing correct synoptics for tissue specimen in place. Autopopulating information already present is one of many opportunities for improvement in pathology efficiency through improvements in the informatics system.
Returning to the daily microscopy rounds, these are great social events for pathologists and trainees to see banter between pathologists about approaching a case. They're daily, timely, quick and opportunity to get differences of opinion. One must be cautious, however, because pathologists may not have time to do a thorough evaluation of the slides or proper observation to render a truly independent opinion. It is important for second opinions to be as independent as possible to protect the first pathologist, the consultant pathologist, and ultimately the patient. They are a social opportunity for sharing knowledge, and for team building and trust. Rounds are not legally documented, allowing for easier discussion and quick review of cases, so should not be referred to as a key component of diagnostic review.
Tumor Boards at Moffitt are a central part of pathologist activity. That's where a team really meets and is a great opportunity to interact with the overall care team. Cancer teams are organized around cancer sites (breast, head and neck, GYN). The whole team, including trainees, nurses, clinical trialists, radiologists, radiation oncologists, surgeons, is there. Pathologist can showcase their depth of knowledge during case review. The oncology team learns from and value well-done rounds.
Rounds should not be overused. Every case does not necessarily need the pathology showing at the rounds. The case should be reviewed prior to rounds, so only education or points of contention are shown by pathologists in depth, allowing for discussion and consideration of additional tests or procedures. One problem is that time limited and a lot of cases need to be reviewed. Time for discussion should be focused on cases that require intense pathology discussion. Sometimes pathologists will get additional medical history findings from attending rounds and learn what the oncologists are really concerned about.
Sometimes pathologists present other pathologists' cases for quality check, and this can cause open disagreement and uncertainty within the department or undermining a colleague's reputation, so disagreements must be managed carefully. I advise further work be done and the pathologists review the case together to arrive at a consensus. A third pathologist could become involved if consensus is not obtained, but really judgment should be used in disagreements for best patient care.
Another problem is lack of complete outside material necessary for making the correct diagnosis, such as biopsies or additional imaging done at other centers. This can be cumbersome, but when discrepancies arise, pathologists should act professionally as a team with the external pathologists, perhaps even using that as a teaching opportunity. We're here to help as pools of experts and as an educational resource to community practices. One should also be aware of poor or variable quality of outside material that could also cause problems with interpretation. Studies may need to be repeated.
Our Pathology and Morbidity Committee reviews errors, near-misses and diagnostic challenges. We try to do this in a blinded way to encourage participation. These problematic cases are opportunities for learning and sharing among the team.
The Safety Committee was created to review critical events and make recommendations. They use a formal process whereby recommendations are written up and go into a policy manual. The departmental chairs and administrators take the recommendations seriously. The Safety Committee will meet a couple of times a month depending on the amount of incidents. It is a 2-year term and generally members do not hold leadership or administrative positions, which keeps it open and makes the committee arm's length from the department administration.
Informatics is a problematic area in pathology, but every problem is an opportunity to build on. Informatics is essential in monitoring the quality of our practice, volumes, case materials, and key performance indices. Systems we use are problematic in that they are very manual, it is hard to program them, and it is hard to extract data we really need. Informatics experts and developers can create a more user-friendly system for the future based on a quality standard because currently many are built on billing and tracking, rather than quality metrics. Moving towards a quality metrics system would improve informatics within pathology. Moffitt's system is not designed to extra with high granularity and we rely heavily on manual extraction to pull information we need to monitor quality practice. Informatics is also used to track outlying cases and workload for pathologists or to build systems for internal quality reviews. I think we will see more use of informatics in the future as an important tool to move pathology practice forward.
Another area in informatics is digital pathology. As it digital pathology grows, it will be a huge opportunity to increase the practice quality through telepathology and through development of new diagnostic assisting tools that run on digital platforms. The future is bright in integrating digital pathology with informatics.
Some challenges to keep in mind
In some subspecialty practices, there may be only one or two experts in that area, so how can we review an expert? The challenge is there are no other experts within the team to review the subspecialty expert. Another challenge is peer reviewing colleagues and how to manage that when colleagues are former students or mentors, creating potential conflict and discomfort. Conflict can grow or cause automatic agreements, which is not the right process for ensuring quality.
Is the team keeping up with practice globally? This is an opportunity to consider proper external consultation practices in finding experts at other sites to share cases with. There is no easy approach to this, although certain groups are looking at creating programs for external proficiency and quality testing, which could be important quality initiatives for the future. Pathologists would be able to share material in an anonymous way with colleagues, focusing on quality practice, and discouraging professional rivalry.
Anatomic pathology ancillary testing may be problematic by nature. It is central to many companion diagnostic studies, particularly in solid tumors. Molecular biomarkers play a key role in personalized medicine and in selecting targeted therapies, such as estrogen receptors and