What happens when your lab fails a proficiency testing challenge? Proficiency testing is, of course, necessary for accreditation, but what happens after receiving a non-passing grade? What needs to be done, and by whom? And why is the process designed the way it is?
To find out more – at least about what happens here in BC – we spoke with Terri McCaskill who is a Proficiency Testing and Quality Control specialist with BC’s Diagnostic Accreditation Program (DAP), the accreditation body for diagnostic services here in British Columbia.
Immediate Impact
So, you’ve submitted your PT results, and the grade came back as unacceptable. You might be wondering whether you’re allowed to continue that sort of testing. The answer is: it depends. In BC, whether your lab can continue testing is up to your lab’s medical director. It’s their responsibility to determine whether the failure is risky enough that testing of this sort should be stopped until a root cause can be identified, or whether the failure is unlikely to affect patient health and testing can continue. Your lab will have its own procedures for addressing this situation, but they should involve your medical director.
Root Cause Analysis
The first step to resolution after a PT challenge failure is a Root Cause Analysis (RCA). A root cause analysis – as the name implies – is an investigation to uncover the ultimate cause of a failure. So, for instance, if the growth media used in a test was contaminated, that might be a cause of the failure, but it probably isn’t the root cause of the failure. The root cause might be the circumstance or set of circumstances that allowed that growth media to become contaminated in the first place, or a flaw in procedures for testing the sterility and viability of new media before testing patient samples. Throwing away that batch of media will solve the problem this time, but it will not prevent the next batch from becoming contaminated.
Critically, a good root cause analysis isn’t about blaming any one person or persons. Someone performing the test may have made a mistake, but what situations led them to make the mistake? Were they trained properly on the equipment they were using? Were they overworked? Did they simply forget some part of the procedure? You might notice a trend here: all of these issues can be resolved through changes in process. People generally show up to work intent on doing a good job. Poorly designed process is what allows errors to go unnoticed or even favor them. The harder the process is to follow, the easier it is to make mistakes.
Everyone makes mistakes, and everyone needs opportunities to learn, grow, and improve.
One practice that can be helpful in removing blame from an RCA is removing proper names from reports related to the analysis and using titles instead (for instance, instead of “Bruce Banner recorded the result in the system”, write “the lab tech recorded the result in the system”). While this approach doesn’t solve the problem altogether, removing proper names can help shift focus away from individuals and onto procedures.
Everyone who participates in testing should be involved in the root cause analysis. Lab techs and quality techs, all the way up to the medical director should be involved. Everyone who has a hand in ensuring quality of test results has a different perspective on the problem and – therefore – different input on its cause and solution. While it’s easy to delegate this work to a quality tech who then asks everyone a lot of questions, a root cause analysis solves problems more completely when everyone involved can be encouraged to approach the problem with curiosity rather than contempt, and to work on the solution together. And adherence to the new procedures will likely be better as well. In the words of Dale Carnegie, “People will support a world they help create.”
Updating Procedures
The work doesn’t stop there. Once a root cause has been identified and a solution has been found, existing procedures and new-hire training have to be updated so problems will go away for good. Without recording and retraining, solutions will quickly fade away and problems will return. It’s important that everyone who needs to follow the new procedures understands them and knows why they have been designed that way.
Patient Impact
Once the root cause has been identified, it’s important to determine whether and how many patient test results were affected by the gap in the process. Not only is this important for patient safety, it’s also a reporting requirement to DAP. These changes in results need to be listed in corrective reports. Patient impact is usually more likely when a problem is systemic.
In BC, this root cause analysis and reporting process is guided by the investigation response form that a lab has to complete for all unacceptable microbiology proficiency testing results. The form is intended to guide labs through the steps of investigating a failure as much as it is to inform DAP of what steps the lab has taken to identify the root cause, corrective actions, patient impact, etc. DAP will review the file, communicate with the lab about any concerns, and store the report in the lab’s file for the next accreditation assessment so the assessors can follow up on corrective actions. If there are exceptions or particularly challenging details in a report, DAP will contact subject matter experts for advice, which can help inform the lab’s response to the problem.
Purpose
Even though it can be a lot of work, this process isn’t about assigning blame or punishing labs for “bad behaviour”. Everyone makes mistakes, and everyone needs opportunities to learn, grow, and improve. These required steps are intended to guide labs through the continual improvement process and make sure that they follow through on lessons they have learned.
Diagnosing and fixing the root cause of a proficiency testing failure can seem like a lot of work, especially for those who are already overburdened.
Yes, proficiency testing is a condition of accreditation, and is designed to test a lab’s ability to accurately test incoming samples. However, thinking about PT in that limited way ignores its grander purpose: to improve quality and lower risk to patients. Diagnostic errors contribute to approximately 10% of preventable patient deaths, and medical reviews suggest they account for 6 to 17% of adverse events in hospitals[1]. Continual improvement has a real impact on lives.
Appeal
For CMPT customers, it’s worth noting that if you have received an unacceptable grade and believe it to be in error, we have an appeals process. When you submit an appeal, our technical committee will review your submission and any supplemental information you provide to decide whether there is cause to change your grade. You can file an appeal for a PT challenge result from CMPT by sending us an e-mail with the details of your appeal. DAP also has an appeals process.
Patient Trust
Living in a country that has a well-regulated medical system, Canadian patients expect that they can trust their test results. And doctors, nurses, and other medical professionals expect that they can rely on those tests to be accurate so that they can make critical decisions about patient care. Every time a correction is issued for a test result, that trust is eroded and the whole system suffers.
Diagnosing and fixing the root cause of a proficiency testing failure can seem like a lot of work, especially for those who are already overburdened. In our last customer survey, 42% of respondents said that their labs experienced stress, fatigue, and overwork, while 32% reported a significant increase in specimen workload. However, it is important to keep in mind that any effort towards preventing a quality failure actually decreases the time, effort, and money the laboratory spends trying to correct a failure.
Joseph Juran demonstrated that internal failure and external failure costs may be 5-20 times greater than prevention and assessment costs, and further, that small increases in prevention and assessment costs can result in a substantial decrease in total costs. This is known as the Juran 80-20 rule.
At the end of the day, though, the work to address PT challenge failures is about keeping test quality high in order to save lives, and we think it’s worth the work.
[1] National Academies of Sciences, Engineering, and Medicine. 2015. Improving Diagnosis in Health Care. Washington, DC: The National Academies Press. https://doi.org/10.17226/21794.
