Page 1
Risk-Based Quality Grades© - An Alternative to Sigmametrics Zoe Brooks ART and Kim A. Przekop MBA MLS(ASCP) CM Awesome Numbers, Inc. Worthington, ON, Canada Rutgers, The State University of New Jersey, School of Health Related Professions, Newark, New Jersey Objectives:
- to introduce the Risk-Based Quality Grade© ( R-B-Q Grades©) as a metric to: a. combine the current risk of the analytical process producing a medically-unreliable result with b. the potential risk of the quality control (QC) process failing to detect a clinically significant error
- to compare the R-B-Q Grades© to Sigmametrics for managing patient risk Methodology:
- We examined analytical process quality and qualit y control practices from three distinct groups separated by dates and QC practice, with a total of 46 laboratories, 274 methods, and 612 QC samples for a total of seven analytes.
- For each QC sample, we gathered: Measured mean; M easured SD; Peer mean; TEa limit; QC chart Assigned mean; Assigned SD; and QC rule(s).
- We used Quality OptimiZer™ software to: a. measure the Current Risk Level as the percent of results that currently fail TEa limits and may be medically-unreliable; b. simulate a shift that would cause 2.3% of results to fail TEa limits; c. recommended a mathematically-optimized 5-part Q.C. process; d. measure the Potential Risk Level as the probability that a patient tested after the simulated error will be exposed to unacceptable risk; e. compare the effectiveness of current and recommended QC processes to detect this significant shift; f. grade patient risk based on current and potential risk; g. measure Margin for Error (to calculate sigma);
- We compared the interpretation of acceptability o f quality and probable action based on R-B-Q Quality Grades© and sigma. Results: Sigmametrics would indicate that 3% of methods require attention; however, R-B-Q Quality Grades© would indicate that 89% of methods have unacceptable potential risk. Conclusions: The Risk-Based Quality Grade© should be further investigated as a metric to manage patient risk. Risk-Based Quality Grades© identified potential patient risk associated with QC design. Risk-Based Quality Grades© should be further investigated as a metric to manage patient risk. ABSTRACT METHODS SUMMARY/CONCLUSION INTRODUCTION RESULTS Quality control can be simplified for laboratory professionals. Software has been developed to grade an assay’s performance based on only a few statistical values. The new R-B-Q Quality Grades© uncovered unacceptable risk in 89% of the assays reviewed compared to only 3% of the assays when Sigmametrics was utilized. CONFLICTS OF INTEREST Zoe Brooks is director of research and innovation at AWEsome Numbers, Inc. and developer of Quality Optimizer™ software. Kim Przekop has no financial interest in AWEsome Numbers or Quality OptimizerTM software. The QC data from the three groups of laboratories was uploaded into the Quality Optimizer ™ software. The data were independently reviewed by both authors, whose conclusions were identical. These statistics were reviewed: current number of data points that fail TEa; TE vs. simulated TEa failure; Margin for Error vs. sigma; current QC practice vs. optimized QC practice; and current patient risk vs. potential patient risk. Risk-based Quality Grades© show that 89% of the methods in the study had unacceptable potential risk levels, while Sigmametrics showed only 3% of the methods had unacceptable potential risk levels. Introducing … Risk-Based Quality Grades© STOP reporting and fix method Is Current Risk acceptable? F Minimum false Positives? Increase QC frequency, increase QC chart review, improve accuracy and/or precision as recommended. Will OptimiZed QC detect failure immediately A B C D Will existing QC detect failure immediately Medical Goal Acceptable Risk as % pts/month Peer Mean (True) Current Mean Current SD Mean on QC Chart SD on QC Chart IQC Reject Rule Q.C. Frequency (n/Month) Sigmametrics R-B-Q Grades Definition: A numeric value that characterizes method performance in terms of the number of standard deviations or sigmas that fit within the tolerance limit or quality requirement of a test. Definition: A grading system A to F that combines [1] current risk of patients receiving medically-unreliable results (MURs) based on measured ability of the analytical processes to meet medical goals and acceptable risk levels and [2] effectiveness of QC processes to detect unacceptable risk levels and [3] opportunity to avoid unnecessary false positive QC flags . Acceptable 3 sigma level is minimum for any business or manufacturing process (66,807 dpm) ( 6.7% long term error rate) Acceptability is based on ability to meet locally-defined medical goals and acceptable risk levels as % acceptable MURs/month. Acceptable goals and risk levels are adjusted to meet clinical need. This 3.6 sigma method illustrates how impossible it is to guarantee error detection in 1 run - even with a 1-2s QC Rule. The inflated SD of current lab practice may take 20 runs. Implement OptimiZed QCP More info on R-B-Q Grades© and their automation: zoe@awesome-numbers.org