Full Version -
A series of news stories and headlines contributed to increased interest by the administration at Moffitt Cancer Center to improve diagnostic accuracy. The lab has taken a leadership role and pathologists have been engaged to identify opportunities to improve quality in ways that lead to better diagnostic accuracy and contribute to improved patient care. Among the quality initiatives underway at Moffitt Cancer Center a project to improve concordance in the diagnoses of surgical pathologists on staff. As part of this effort, there is also engagement with pathology groups at community hospitals associated with, and referring patients to Moffitt. This presentation will look at the challenges of creating a system to better assess the quality and consistency of diagnoses produced by individual pathologists, the goals being to improve the processes in anatomic pathology so that patient safety is enhanced, and improved diagnostic accuracy contributes to better patient outcomes.
- Understand the challenges of creating a system to assess the quality and consistency of diagnoses produced by individual pathologists.
- Be able to set goals to improve the processes in anatomic pathology so that patient safety is enhanced.
- Understand how improved diagnostic accuracy contributes to better patient outcomes.
ANTHONY M. MAGLIOCCO, M.D., FRCPC, FCAP: Thank you very much for that kind introduction, and thank you for inviting me to share my experiences with quality anatomic pathology. I think that this is a really interesting time for anatomic pathologists, and also a challenging time. So I’ll try to move through the presentation fairly quickly for you.
The objectives of today’s talk is to really review the challenges in anatomic pathology, to share with you the type of system that we’re implementing to Moffitt Cancer Center, and to really discuss how improved quality can contribute to better patient outcomes.
Now we live in changing times as practitioners in medicine, and we’re moving from a reimbursement model where we are paying for service to one where payers are now demanding quality that are the investments in the system actually paying off in improved outcomes for patients. In addition, with internet and broadly available information, the patients are becoming more empowered about what is happening to them. And with the changes in reimbursement and insurance, patients are facing higher deductibles.
So they are also taking greater ownership in how they are selecting their treatments. As information is coming out into the public domain, there is a reduced tolerance of error. In our society, error is not really expected. And we will review what these changes mean for anatomic pathology.
Unfortunately, for anatomic pathology, not all news is good news. And one thing about pathologists that I think is that pathologists tend not to really market themselves very well. So often the type of marketing that they do get in the public eye is frequently negative where when problems affect patients, the patients sometimes go to the press, the press looks at investigations, and we see a number of studies now coming out where basically there is not really good coverage of pathology mistakes, problems with pathology, and things encouraging patients to seek second opinions.
It’s also important to remember that today’s system is getting more complex, and pathologists don’t just make diagnoses. They actually make recommendations about therapy. The staging of a cancer can effect what type of this used, and the selection of biomarkers and the performance of those biomarkers in a path lab can also impact which treatment is used, whether something like Herceptin is indicated for a patient or not.
Now the other thing is that there is a lot of information for patients now for the internet. And what patients face a diagnosis of cancer, frequently they will start browsing the web. There is one website called Cancer Points where they present information for the patients to help them understand what their options are, what are the implications of cancer and so on. And on this website, they have highlighted a number of studies, including a study from Johns Hopkins and others that show that as much as 44% of the time, pathology reports may contain errors. This high rate of error refers to cases of prostate cancer with incorrect Gleason grading which may affect the treatment choice for a particular patient.
Now we also recognize that pathology is still practiced by pathologists, and pathologists have different training backgrounds, different abilities, different skills, and often they are still practicing in isolation and in silos. And as this becomes apparent, again, the media is now identifying this that the quality of care that the patients are receiving at different centers are different. In the top, this is a study from Canada where it was determined that estrogen receptor determination in a single lab was really incorrect for a period of over ten years.
In another study, again, in Canada, in Newton, New Brunswick, it was discovered that a practitioner in New Brunswick may have been making errors for as long as 13 years. So these kinds of discoveries have certainly affected health systems in determining what is happening in pathology, is there a real problem in anatomic pathology, and how are we going to address that.
Now if we look at the other side in terms of litigation, where the litigation problems are arising, so we can break down the anatomic pathology process into three steps, basically pre-analytical, analytical, and post-analytical, most of the litigation is actually in the analytical area which is the active diagnosis. I sometimes think that in the pre-analytical, that frequently errors in pre-analytical may not actually be detected, and I’ll talk a little bit about that as we go through the talk.
So I believe that anatomic pathology has a great opportunity for improvement in quality. Now if we look at where anatomic pathology begins, this is an example of a gross room in a typical hospital. And we see that this is somewhat of a chaotic environment. The anatomic pathology lab looks quite different from a chemistry lab or a blood bank in that we are managing tissue, that the environment is a little bit more chaotic, and it’s not automated at this point in time.
The problem is that this brings up issues with potential identification of specimens or are the specimens identified correctly.
And as the process of managing these specimens occur in the labs, there are many steps where specimens are transferred. They go from containers to the examination, into cassettes, and there are opportunities for errors and mix-ups to occur at every possible step. Going back also with material like this, it’s very important to maintain a clean environment, and also with instruments because material can get stuck on instruments and be carried over from one case to the next causing biopsy mix-ups, and floaters that can confuse diagnosis.
This is another hand-off where material comes from for embedding, and then basically additional hand-offs where blocks are cut─
Material is floated in a water bath, and this is another area where potential problems do happen that there may be residual pieces of other specimens that can potentially get mixed onto a specimen as it gets mounted onto a slide. So that is the pre-analytical process, and I think that there are opportunities for improvement in pre-analytical pathology and anatomic pathology in terms of minimizing the opportunities for a mix-up of specimens and tracking specimens through that process.
Now the specimen finally makes it to your friendly neighborhood pathologist, and there is a smiling Dr. Kallele [phonetic], a friend of mine in Calgary, who is a great pathologist. And, again, the pathologist’s office is not changed a lot over time in that they have complex filing systems and that the real diagnosis rests on the pathologist. So the pathologist is very key to how a patient is diagnosed and how a specimen is reported out. Now one thing about pathologists as it was noted, the pathologist is perhaps the most important physician that the patient never meets.
So often, a patient is unaware of what the pathologist is going to receive, what is the training of the pathologist, what are the skills of the pathologist. And in a complex system like ours, really it’s incumbent upon us to ensure that all of the patients receive the highest possible level of care. So that means we have to come up with a system where pathology knowledge is available and shared and made available for really every patient that comes through the system. Now the purpose of quality assurance in anatomic pathology, clearly the baseline is to comply with accreditation and certification. But I view this as really a baseline and that should be a given. We want to make a system that goes beyond what the minimal expectations are to a total quality system where we really can minimize errors and ensure safety and quality for all of our patients.
Quality in AP
And this isn't just to avoid litigation. It is really the right thing to do and it will improve efficiency and quality.
Now if we look at interpretive diagnostic errors where interpretation comes in, there was a meta-analysis done by CAP that showed that there were a number of studies, so they actually reviewed 89 studies. And in those studies they found that the error rate of discrepancies was as high as 18% in surgical pathology. And major discrepancies in what they defined as a major discrepancy are discrepancies that would affect treatment. They reviewed 66 studies and 6.3% of them showed major discrepancies.
Now if we look at breaking it down into whether cases were reviewed internally versus externally that it was determined that if you had an internal review, the discrepancy rate was only 1.3%, whereas if you had an external review the discrepancy rate was much higher at 7.4%. And that may not be too surprising, but we will walk through why that may be the case.
Now one thing that did affect us at Moffitt is that there was a publicity of the case where there was an error made in the diagnosis and that did trigger Moffitt to look at its processes. Now just to give you a little bit of a background of the Moffitt Cancer Center, the Moffitt Cancer Center is located in Tampa, Florida. It’s the largest cancer center there. It’s a comprehensive cancer center, and the NCI designation for comprehensive cancer centers is granted to centers that really focus on cancer care and have integrated programs of both treatment and research. I believe there are only 46 of them now in the United States.
Moffitt is a busy practice for a cancer center seeing over 17,000 new patients a year, and basically over 300,000 outpatient visits a year. The practice pattern of Moffitt is multidisciplinary, so it’s not organized into traditional departments except for pathology, radiation, oncology, and radiology. The other teams are organized around tumor sites.
And the other thing that Moffitt runs is something called Total Cancer Care, which is a very deep research program to essentially try to enroll as many patients as possible with their permission to enable us to study how the patients are treated, the molecular aspects of their tumors, and outcome. And to-date about 80% of the patients agree to enroll in this program, and we currently have over 100,000 patients enrolled.
Now the value of this program is that it creates a rich research database for us to understand how treatment evolves over time, it enables us to look at these tumors to test new biomarkers to determine how they might impact─ how testing might affect the measurement, and to look at how the patients are treated and with outcomes, so there is a fairly rich resource here at Moffitt that allows us to do research in this particular area.
The other thing that Moffitt has moved towards is something called Pathways of Care. And this is where essentially Moffitt has created pathways to facilitate or to enable oncologists to understand what treatment options are available and to help them select the best ones. These pathways were written by Moffitt experts in each field. They were written after extensive consultation with NCCN guidelines and a review of the literature, and a review of current practice. And these are living documents that are updated on a regular basis that enable us to try to standardize care.
On the Moffitt website, they are all available, so for each tumor site a number of pathways are available. And so, for example, under breast, there are pathways for treating invasive breast cancer and also for diagnostic evaluation.
Now this is what the pathway looks like in the programming sense. It is displayed to the oncologist in a different way, but essentially what happens is that there are decision points identified for the patient. The oncologist is notified if there are clinical trials available. And one other aspect of this is actually the cost of the treatment that is also displayed to the oncologist so they might consider alternative treatments and really understand what the impact of a particular treatment has for a patient. There are also surveillance recommendations included in these pathways.
Now Moffitt also has a deep program in personalized medicine, and this personalized cancer medicine is integrating genomic testing, complex testing, and it centers around having a clinical genomics action committee, and the personalized medicine consultation service. Now these services are essential to ensure that the correct test is being used, and also that the results are available for the oncologist and the patient in an understandable form. Often we do find opportunities for off-label use and clinical trial use, and we have found that recommendations from these committees go a long way to helping insurers pay for those treatments as they do respect the recommendations of these multidisciplinary evaluations.
Now we will go over the review process that we currently have in anatomical pathology at Moffitt to share with you so that perhaps you may use them in your own practice. The first thing that happens is that all new pathologists go through a focus review. So when a pathologist comes on at Moffitt for a period of six months, they are guided by the more experienced pathologist and their cases are reviewed to determine if there are any issues in their practice.
We have a policy of second expert review in all new Moffitt patients, so all patients that come from outside do have a review and a presentation at the multidisciplinary tumor board rounds which we feel is very valuable to ensure that the report is integrated and that the oncologist has the latest information available. We also have created policies regarding consultation so that if there is a major error detected that there is a way to manage that error to report it back to the oncologist and the patient, and then also to the regulatory within the hospital.
We mandate the use of synoptics because we feel that that ensures complete reporting, and also enables us to capture discreet elements for monitoring. The key point in any sort of quality program, you have to be able to measure your activities and ensure that you can measure improvement as you implement change. Another important aspect is the daily microscope rounds where cases, difficult cases, are shared amongst the pathologist. And there is a process for formalized interdepartmental consultation so that those are actually tracked and monitored.
Another important activity is a week or a monthly quality assurance meeting which is actually mandatory for all staff to attend. And I will show you the aspects of that as we move through the presentation, but the data is shared with all of the pathologists about their performance and about the performance of the entire department, so it is important that that information does get back. We are continuously developing key performance indicators. Now for a key performance indicator, it is important that the indicator is actually easy to collect and that it is meaningful.
And I don't think that you need a lot of them, but you have to think carefully about what they might be. They could be things like turnaround time, they could be things of a major change in diagnosis, lost specimens, etc. Those are important to establish and to monitor. We also have a special rounds called our morbidity rounds in pathology. And this is where misses, near misses are brought for discussion so that we can learn from errors that do occur in the department so that we can try to avoid them in the future.
Another thing that is important is the safety committee. I’ll talk to you a bit about that again to emphasize that. That is the committee that meets ad-hoc to review all incidences as part of the critical incident review. And another thing that we are really investigating now is external proficiency testing reviews and the potential use of telepathology to really access experts beyond the walls at Moffitt to really ensure that our practice is integrated nationally and internationally.
The other mechanism that we do have to try to modify practice is quality incentives. And we can tie quality; quality is a component of the annual bonus program at Moffitt, usually a different quality indicator is selected each year and monitored for the faculty.
Now going through this in a little bit more focus. We view pathology, we feel it’s essential for all new cancer patients. We saw that there is a significant discordance rate in the community practice versus the centralized cancer lab practice. This doesn’t mean that the community pathologists are poor practitioners by any means, it’s that the type of practice is different, and that the typical community practice may be as few as one to four pathologists, and it’s very hard for them to maintain expertise in a rapidly changing cancer world where there are new advances continuously.
In a practice like Moffitt, we do have 25 anatomic pathologists and 13 hemato-pathologists all structured into a subspecialty practice enabling the teams to really focus on specific tumors of interest and develop very deep expertise in those tumors. In addition the integration of the pathologists with the multidisciplinary team gives them significant clinical insight to understand how pathology impacts clinical decision making.
We recommend that expert review is performed before commencing definitive treatment so that the patient is on the right treatment path right from the beginning.
Again, this is a subspecialty practice that enables a really deep understanding of each tumor site. And there are also opportunities for academic development and training in this environment.
Synoptic reporting is something that is becoming more mandated. I think it has some positive and negative aspects. The positive aspects are that it does help the pathologist to ensure that reporting is complete so that there are reminders; yes, I do need to report about this margin, I do need to measure this thing, the size of the tumor, I do need to report grade, and I do need to order certain biomarkers. So it does ensure completeness.
But what it does not ensure is accuracy because essentially that relies on the pathologist’s interpretation. When it’s reported back to the oncologist and the patient, it does give a very good illusion of accuracy because the report is so detailed and has so many aspects to it. But we should remember that completeness does not necessarily equate with accuracy and we need to be certain that what is being reported in the synoptic is actually correct.
The daily microscope rounds are a great feature for a big department. It enables the pathologist to meet with their colleagues to share difficult cases, and also to share cases across subspecialties because the pathologists do cover the frozen sections where multiple specimens are seen, so it enables them that even though they are focusing on one subspecialty, to maintain to see cases from different specialties as well.
It also allows the pathologist to calibrate their calls of atypia, etc, on cytology, and to share knowledge. One problem I have noted with these rounds is that sometimes they are not well documented and they may be rushed. So if you are doing these rounds you have to think about that. Are you actually making clinical decisions by showing one slide out of 30 to your colleagues? Is the decision thoughtful, are you going to document in some way and include it in a report?
So I would discourage using the multidisciplinary route to really make decisions, and if a critical decision is required, that a formal intradepartmental consultation is probably indicated so that it can be─ so that the consultant has ample opportunity to really review it in depth and write a detailed report.
Now tumor board activity is very busy at Moffitt. We have 13 tumor boards a week, so there are two a day. Pathologists are assigned to these, and they can review as many as 30 or 40 cases per tumor board. So they are busy. It does allow the pathologist to interact with the oncologist and enable them to understand how pathology diagnoses impacts treatment recommendations.
Some of the problems with tumor boards is it can be difficult getting outside materials, and the quality of the outside materials can be variable and can be difficult to read at times.
Our morbidity and mortality committee meets as needed generally once a month, and reviews all of the errors, near misses, and diagnostic challenges. And it’s done generally in a blinded way so to avoid putting significant embarrassment on pathologists and to enable an open discussion of issues so that we can really share our potential errors with each other and try to learn from that so that we don’t make them in the future.
We formed a safety committee two years ago. We have picked general faculty so these are members that are not holding any administrative positions. They are practicing members. If there are specific cases, sometimes they may call ad-hoc members. They review critical safety events and they make recommendations. It’s recorded, it’s a formal process, and they can meet on an ad-hoc basis. And we have found this committee quite useful. There are about eight members on it and they give great insights when errors occur and to process improvement.
An area of concern or an area of opportunity I think for pathologists is informatics. And typical pathology systems are things like Cerner. And I have discovered that these systems are really designed for billing and reporting. They really lack the tools necessary for adequate quality assurance monitoring, and that is problematic. It’s difficult getting key performance metrics out of these systems.
And we are really struggling with that. Often we use a lot of human man hours to search for data to be able to monitor our practice. And I put this out as a challenge to the IT and the information system providers to give us better quality assurance metrics.
So once a month we do review all of our cases. We look at different types of errors that occur in terms of whether, for example, typos occur, if specimen’s are─ if reports showed an error and so on, and we monitor this on a monthly basis. So these are important metrics, SP or internal surgical cases, shows our caseloads, and essentially our potential error rates that we’re seeing internally.
The other thing is we monitor synoptic use, so these are each of our pathologists, to determine if the pathologists are using synoptics in a way that we mandate. And if they are falling short, we can identify those pathologists and determine why the synoptics are not being used and help them incorporate that into their practice.
We also have a significant, or we look carefully at case correlation. And we look at two components internal correlation events looking at the basic discrepancies between each of our programs. And we can see that there are a number of discrepant cases that do turn up. Certain programs have very high rates of changes in diagnosis from the community practice. We notice in neuro-path, as many as 45% of the cases are altered. In other tumor sites, for example in GU, as many as 20% of the cases show alterations on review.
We also track occurrences that need scrutiny such as lost specimens, cancelled orders and other problems, so reviewing these on a monthly basis helps our team understand how these errors occur and bring up systems to improve them.
Now what are the key challenges? It’s great to put on an internal review program, but the challenges even in a big practice like ours is that we only have one to three experts in each subspecialty practice. And it creates a challenge of reviewing your peer because you may be reviewing a case that actually came from your mentor, maybe a case from your friend, it may be a case from your rival, and this is a potential problem in that in human interaction, it’s difficult to really give a very critical review in those instances.
And this can turn into rubber-stamping. And this may explain why internal reviews turn up less errors than external reviews. And I think that as a pathology practice evolves, we really need to look at systems where we can get broader external reviews so that we can get unbiased reviews, blinded reviews from people that are at arm’s length from our own practice. And, you know, certain opportunities are now rising to enable us to do that.
Other Quality Issues
Another just quick review of other aspects of quality in anatomic pathology is that anatomic pathology frequently provides important ancillary tests that guide treatment recommendations.
These include molecular biomarkers. The other thing about anatomic pathology is that tissue was never really designed to be really well analyzed for biomarkers as we saw the way that growth rooms were set up.
Now there is a program in Canada, the