Our vision: Better patient care through better QC

June 2016:
We see a world where everyone on the quality/risk management team shares a common language and works toward a clearly defined common goal. For every QC sample that represents patient risk, medically allowable error limits, and the acceptable number of results that may fail those limits, are approved by “The PIPS” (Patients, Institutions, Physicians,and Society.)

Practically, this is achieved through each lab’s laboratory director and medical advisory committee. Once those numerical quality standards are set, factual risk drivers can be converted to pass/fail evaluations, action flags, projections of the number and cost of laboratory errors. M.O.R.E. Quality software evaluate specific factual risk drivers to determine if risk is acceptable and to consistently guide interpretations and action.

Ten years ago, Zoe Brooks wrote a series of articles on The Q.C. Gap. The actions recommended in the final article below are finally within our grasp – thanks to risk management concepts and the relentless effort of many researchers, authors, educators and caring laboratory professionals.

Change takes time, effort, education, collaboration, determination.

Quality assurance
What can you do to close the QC gap?

Oct 2006 Zoe Brooks

This is the forth and final essay in a series of four essays on www.bloodgas.org. (Now acutecaretesting.org/en/articles/what-can-you-do-to-close-the-qc-gap)
The first essay, “Quality control in theory and practice – a gap analysis“, raised the question: Has “the system” given front-line laboratory workers the knowledge and tools they need to make quality control decisions wisely? Or is there a significant gap between QC theory and QC practice at the front line?

The second essay, “I found the gap… it’s in the basement!”, provided examples and exercises to support my fear that “the founding principles of laboratory quality management are often poorly understood, inadequately practiced and inherently flawed”.

The third essay, “Quality control… the gap deepens“, examined highlights from one of the online quizzes to see if this informal sampling supported the existence of a gap between QC theory and QC practice. It also raised the issue of a deeper and more insidious gap – a gap between perception and reality.

Each previous essay ended with: “These are my observations, and I truly hope that many of you will stand up and prove me wrong. If you would like to discuss this essay, or test your quality savvy with online quizzes, log on to [site obsolete]

We are now at the forth and final essay – and no one has yet attempted to stand up to prove me wrong. So the question remains: “What can you do to close the gap?”

When the editorial board at bloodgas.org agreed to publish this series of articles 18 months ago, it was with the condition that the final essay be “What can you do to close the gap?” I figured that gave me lots of time to come up with a solution. This series of articles and the analysis of quiz results from people all around the globe has been a truly fascinating exercise. It has left me even more convinced that a serious gap does exist between “what should be done” and “what is done” in the critically important field of laboratory quality control, and someone should do something about it.

“What can you do to close the gap?”

What you can do to close the gap depends on who you are and where you can influence practice.

HERE'S WHAT I WOULD DO... IF I WERE WORKING IN A CLINICAL LABORATORY
If I were a member of the front-line staff
  1. I would make sure that I understand the concepts of quality control.
  2. I would dare to ask “he/she who cannot be questioned” to explain what each number means and where it comes from, and what is OK, and why that is OK, and what must I do if it is not OK.
  3. I would regard each patient sample as if it came from a beloved friend or relative, and make sure that when I initial a report as OK to go to the clinician, that the results are of suitable quality to enable him/her to reach an appropriate decision.
  4. I would take pride in the role I play in this very important profession.
If I were a senior scientist

I would do everything the front-line staff member does, plus

  1. I would ask the staff reporting to me:
    1. What is the right answer for a specific QC sample?
    2. Where does that right answer come from, and
    3. When and why would that right answer change?
    4. How close do the QC results have to be to the right answer for patient results to meet the needs of local patients and clinicians? (What are the acceptable limits?)
    5. Where do those acceptable limits come from?
    6. How would you know if results are not good enough to report?
    7. What would you do if results shifted from x to y?

Oh, by the way, I’d make sure I knew these answer myself first!

And I would make sure that what is done in the laboratory is the same as what is written in the procedure manual – and that those written procedures reflect current scientific best practices.

If I were a laboratory medical director

I would do everything the front line staff member and the senior scientist do, plus

  1. I would communicate with the medical staff in my facility.
  2. I’d ask the clinical staff “If a patient sample had a true value of ‘x’ for ‘test A’, at what value would you consider that the patient’s condition was changed and take a different clinical course of action?”
  3. I would compare those answers to the allowable error limits recommended in the literature, and use it to set my internal performance standards.
  4. I would discuss the concepts of biological and analytical variation with the medial staff and tell them the size of variation that the lab can reliably detect.
  5. I would pressure my LIS company to produce reports that couple known information on biological variation with my current quality control data on bias and imprecision to give the clinician an indication of the probability that the difference between two results reflects a true change in the biological condition of the patient.
  6. I would conduct studies to confirm that the reference ranges reported in my laboratory reflect the patient population we serve.
  7. I would make sure that the quality control processes used in my lab are:
    1. based on samples that do indeed reflect changes in patient samples
    2. capable of correctly reflecting method accuracy and precision
    3. clearly defined in procedures for all staff
    4. validated by competency assessment of staff at all levels
If I were a quality assurance manager

I would do everything the front-line staff member and the senior scientist do, plus

  1. I would ensure that the processes and performance standards approved by the medical director are clearly documented and properly implemented by all staff.
  2. I would challenge staff with quizzes and sample scenarios to ensure that unacceptable patient results will be quickly identified and corrected.
  3. I would encourage a culture of quality.
  4. I’d post results graphically and celebrate meeting performance standards.
  5. I would encourage staff to identify OFIs (opportunities for improvement).
  6. I would challenge my LIS and/or external QC software to ensure it is performing as recommended by scientific experts.

Compare the 'tree' above to the 10 year old version below.

There are subtle but significant differences between the ‘Tree of M.O.R.E. Quality,’ above, and “The Dysfunctional Christmas Tree,” below.

In the new version, there are four essential roles and competency sets.

  1. At the top, regulatory and professional associations (the people who make sure quality standards are met) are grouped directly with “The PIPS” – Patients, Institutions, Physicians and Society – (the people who define and/or approve quality standards.) They need the competencies of “Risk Assessors” to answer “Is Risk Acceptable?” Others must do M.O.R.E..
  2. Lab directors and quality managers require the skills to use software to evaluate risk, verify QC effectiveness and optimize QC processes.
  3. Front line staff and supervisors who handle patient samples and decide many times daily “”Is Risk Acceptable?” are Risk controllers. It is ultimately their action that determines patient risk. They need clear processes and QC software they can trust to alert them if risk is unacceptable – and to recommend action. Their competencies focus on response to action flags.
  4. Manufacturers and educators are included with the Risk Masters. They need to understand the math and logic. They are provided with verified core ‘software apps’ and competency programs. They can add and individualize functions, reports and material on top of the core ‘operating system’ that evaluates factual risk drivers. Thought leaders work directly with educators and enablers. Concepts and software pass academic scrutiny and practical implementation tests before introduction. By moving the thought leaders to the roots of the system, and providing verified capabilities and programs to manufacturers, valid scientific processes will be clearly taught in educational institutions and practiced by laboratory staff and manufacturers.

This is the future we see.
 

Remember! Both these apply only to tests with numerical results and stable QC samples.

FIGURE 1 (below – from the 2006 article) shows a diagram of most of the key players in the laboratory industry. I could not draw an organizational chart or trace relationships. In theory, the wisdom of the experts should trickle through the entire laboratory industry and permeate every aspect of what we do. In practice, I do not see a process to ensure that valid scientific processes are clearly taught in educational institutions and practiced by laboratory staff and manufacturers. In my experience, regulatory requirements and inspections do not adequately verify that quality practices are referenced to scientific best practices.

In theory, the wisdom of the experts should trickle through the entire laboratory industry and permeate every aspect of what we do.
HERE’S WHAT I WOULD DO… IF I WERE WORKING IN OTHER CRITICAL AREAS
If I were in charge of a proficiency testing program or EQA program

I would draw on the expertise of the full range of laboratory professionals to create surveys and case studies that challenge the quality control process.

If I were in charge of legislation governing requirements for laboratories

I would require laboratories to prove that their quality control practices can be traced to scientific references. I don’t mean that they show the correct reference at the end of their written procedures, I mean that they are actually doing what is recommended by scientific leaders.

If I were in charge of a college or university program teaching preliminary or advanced programs for medical technologists or scientists
  1. I would ensure that students are taught the fundamental meaning of quality, and how quality standards are set for clinical laboratories.
  2. I would ask the teachers who prepare and present quality control education to show me how their material clearly teaches the students the meaning of the numerical data involved.
  3. I would insist that students be able to describe in words the information conveyed by each number.
  4. I would require that all educational material be referenced directly to scientific recommendations.
  5. I would require tests and case studies that clearly demonstrate competence – the ability to recognize and react appropriately to instances of unacceptable quality.
  6. I would fire anyone who still wanted to teach that “As long as your results are within ±2 SD, everything is OK.”
If I were in the person who signed off on the final examination process to certify medical directors or other laboratory professionals
  1. I would make it my personal obligation to ensure that the certification process requires graduates to be competent in the application of current recommended quality management processes.
  2. I’d make sure that professionals in charge of safeguarding patient wellbeing would practice in a laboratory ONLY after proving they can assess QC data and charts to decide if a system is operating well enough to meet patient needs.
If I were in charge of a commercial peer comparison program
  1. I would ask a laboratory statistician to check my math to ensure that all the numbers I report are accurate and meaningful.
  2. I would explain exactly how each number is calculated and what it means.
  3. I would relate the numerical reports for each sample to clinical need or performance standards, in addition to peer comparison.
If I were a manufacturer of quality control material
  1. I would conduct and publish studies proving that my QC samples will indeed shift higher or lower if a change in the analytical process causes patient samples to shift higher or lower.
  2. Similarly, I would prove that the precision reflected by my QC samples reflects the precision of patient samples.
If I were a manufacturer of laboratory instrumentation and/or reagents
  1. I would conduct and publish studies proving that my system is capable of meeting defined and referenced performance standards.
  2. I would clearly define how the user can adequately monitor ongoing accuracy and precision, using methods referenced to current scientific recommendations.
If I were a QC software developer
  1. I would publish a certificate that a recognized laboratory quality control expert has verified that my quality control practices are indeed the same as those recommended in scientific references.
  2. I would challenge the system to ensure that all possible quality control flags will be generated as planned.
  3. I would make sure that people cannot do things like mix data from different reagent batches or instruments, or “turn off all the Westgard rules so they can report results without flags”.
  4. I would create action reports to make it easy for senior staff to constantly monitor performance against defined standards.
If I were in charge of CLSI, or similar organizations

I would develop implementation plans and teaching guides to accompany each guideline.

If I were a scientific expert who creates and publishes quality control concepts
  1. I would rewrite the scientific literature into clear, simple, straightforward language with detailed examples for each discipline
  2. I would provide written sample procedures with clear instructions on exactly how to obtain each input number, and interpret each output number
  3. I would test what it takes for others to become competent in the applications of my concepts, and provide appropriate teaching tools for all levels of lab professionals
If I could, in any of the above “personalities”
  1. I would create an Internet community where laboratory professionals can come together to share best
  2. practices and benchmark their analytical performance against performance standards.
  3. I would promote a culture of confidence, competence and pride.
  4. I would certify laboratory professionals at various levels based on competency challenges.
  5. I would host quality management Olympics.
  6. I would applaud those who take steps forward to better quality.
If I was one of those folks, I would do whatever I could to close the gap. But I am none of the above. So here’s what I would do … if I was just me, or maybe you

I’d start talking about quality.
I’d let people know that there may be a problem here.
I’d look for opportunities to make things better.
If we start talking, we might be able to open communication channels so the recommendations of the experts become everyday practice.

CONCLUSION
I wish to thank the editorial board at bloodgas.org for providing a forum to explore this potentially controversial topic, and also the many people who visited my website and challenged the QC quizzes. I am delighted that this series of essays has raised awareness, and will apparently make a difference. (published Oct 2006)
Once again, these are my observations, and I invite you to discuss this essay or test your quality savvy with QC quizzes by logging on to [site obsolete]

Watch for a future essay to compare the recommendations of experts and provide a checklist to verify your QC processes!

 

REFERENCES

  1. Online quizzes and discussions at [site obsolete]
  2. Westgard JO, Burnett RW, Bowers GN. Quality management science in clinical chemistry: A dynamic framework for continuous improvement of quality. Clin Chem 1990; 36; 1712-16
  3. Fraser CG. Biological variation and quality for POCT. bloodgas.org, 2001
  4. Klee GG. Quality management of blood gas assays. bloodgas.org, 2001
  5. Westgard JO. Quality planning and control strategies. bloodgas.org 2001
  6. Westgard JO. A six sigma primer. bloodgas.org, 2002
  7. Bais R. The use of capability index for running and monitoring quality control. bloodgas.org, 2003
  8. Kristensen HB. Proficiency testing versus QC-data comparison programs. bloodgas.org, 2003
  9. Thomas A. What is EQA – just another word for proficiency testing? bloodgas.org, 2004
  10. Ehrmeyer SS, Laessig RH. The new CLIA quality control regulations and blood gas testing. bloodgas.org, 2004
  11. Tonks DB. A study of the accuracy and precision of clinical chemistry determinations in 170 Canadian laboratories. Clin Chem 1963; 9: 217-23
  12. Westgard JO, Quam EF, Barry PL. Selection grids for planning QC procedures. Clin Lab Sci 1990; 3: 271-78
  13. Fraser CG, Kallner A, Kenny D, Hyltoft Petersen P. Introduction: strategies to set global quality specificationsin laboratory medicine. Scand J Clin Lab Invest 1999; 59: 477-78
  14. Brooks, Z. Performance-driven quality control. AACC Press, Washington DC, 2001 ISBN 1-899883-54-9
  15. Brooks, Z. Quality Control – From Data to Decisions. Basic Concepts, Trouble Shooting, Designing QC Systems. Educational Courses. Zoe Brooks Quality Consulting. 2003
  16. Brooks Z, Plaut D, Begin C, Letourneau A. Critical systematic error supports use of varied QC rules in routine chemistry. AACC Poster San Francisco 2000
  17. Brooks Z, Massarella G. A computer programme that quickly and rapidly applies the principles of total error in daily quality management. Proceedings of the XVI International Congress of Clinical Chemistry, London, UK, AACB 1996
  18. Brooks Z, Plaut D, Massarella G. How total error can save time and money for the lab. Medical Laboratory Observer, Nov. 1994, 48-54
  19. Brooks Z, Plaut D, Massarella G. Using total allowable error to assess performance, qualify reagents and calibrators, and select quality control rules: real world examples. AACC Poster, New York, 1993
  20. Brooks Z, Plaut D, Massarella G. Using total allowable error to qualify reagents and calibrators. AACC Poster, Chicago, 1992

 

AUTHOR
Zoe C. Brooks
8070 Highway 17 West
Worthington>Ontario
Canada P0M 3H0

 

What can you do to close the QC gap?