Page 1
Total Error as Percent [(Measured Mean - Package Insert Mean) + 2xSD] as % of Package Insert Mean (submitted as true value.) TE is negative when bias is negative. ALP CLIA = 30% BV = 11.7% Is risk acceptable? Lab 1 Lab 2 Lab 3 Lab 4 ALT CLIA = 20% BV = 27% Is risk acceptable? Lab 1 Lab 2 Lab 3 Lab 4 Urea CLIA = 9% BV = 15.5% Is risk acceptable? Lab 1 Lab 2 Lab 3 Lab 4 Cast Your Vote Yes No 39,699 MUR/Yr. $396,990 0 5,000 10,000 15,000 ALP -1 ALP -2 ALP -3 ALP -4 ALT-1 ALT-2 ALT-3 ALT-4 Urea-1 Urea-2 Urea-3 Urea-4 MUR/Yr. vs. Biological Variation goals at modeled (not real) 300 patients/day 5,158 MUR/Yr. $51,580 34,597 MUR/Yr. $345,970 0 5,000 10,000 15,000 ALP -1 ALP -2 ALP -3 ALP -4 ALT-1 ALT-2 ALT-3 ALT-4 Urea-1 Urea-2 Urea-3 Urea-4 MUR/Yr . vs. CLIA goals at modeled (not real) 300 patients/day 5,621 MUR/Yr. $56,210 Cost of MUR/Yr . in 4 labs @ 300 pt/day; Total patient volume = 432,000/year; Cost arbitrarily set at $10USD/MUR Analyte BV CLIA USD $451,698 $405,856 Egyptian Pounds £ 40,652,819 £ 36,527,035 Cast Your Vote: Which goal is BEST? CLIA Limits Biological Variation Medical Goals CLSI EP 23-A states that risk is acceptable when it is “small enough such that patients, physicians, institutions, and society are willing to risk the consequences.” This study was designed: 1) to evaluate acceptability of risk in four laboratories for urea, alanine aminotransferase (ALT), and alkaline phosphatase (ALP); 2) to identify the predominant source of faults; and 3) to compare the effectiveness of existing QC processes to those recommended ‘Mathematically-OptimiZed Risk Evaluation .’ Each lab set TEa limits according to common practice. We set the acceptable current risk level at 2.275% to reflect the practice of considering a 2-sigma method acceptable. We calculated current risk as the percent, number and existing cost of medically-unreliable results (MURs) from sigma values for each QC sample based on verified QC data and patient test volumes. We assigned the average cost of errors reported at $10US (90LE) per MUR. We applied the process of Mathematically-OptimiZed Risk Evaluation to assess the acceptability of risk, identify faults and advise action. Seventeen of the 24 QC samples had error rates less than 2.275%; 10 samples reflected methods currently producing <1 medically- unreliable result /year; and 7* samples failed the TEa limits set. Selected total allowable error (TEa) limits varied from 11.7% to 30% for Alkaline Phosphatase. This practice is incongruous with recommendations to set medical goals. Evaluation of risk and recommended action varied between laboratories and between analytes. Faults creating imprecision were more common than those causing imprecision. We concluded that the calculated risk level is generally not accepted in healthcare. Adjustment of these procedures is mandatory. Mathematically-OptimiZed Risk Evaluation detects laboratory risks that are missed by commonly used statistical QC. Zoe Brooks is CEO and Director of Innovation and Research at AWEsome Numbers, Inc. Kim Przekop has no financial interest in the company. Acceptability of Current Patient Risk Brooks, ZC ART 1; Przekop, KA MBA MLS(ASCP) CM2 ; El Sharkawy, RM MD 3; El Hadidi, ES MD 4; Elattar, EA MD 5; Kandil, NS MD 6; Shaheen, EAH PhD 7; Ibrahim, OA MSc 8; Wagdy, B MSc 9, Farrag, E MSc 10 1,2 AWEsome Numbers Inc., Canada; 3,5,6,8; Medical Research Institute, Alexandria University; 4 Ein Shams University; 7 Alexandria University - Egypt ; 9,10 Private laboratory 6 of the 24 QC samples had sigma values less than 2.0 compared to CLIA goals. 7 samples had sigma values less than 2.0 versus biological variation goals. There were only 2 QC samples from 1 analyte in 1 laboratory that failed vs. both TEa limits. 8 MUR/Yr. $80 313 MUR/Yr. $3,130 Contact Information: Kim: Crosswayzconsulting@gmail.com Zoe: Zoe@awesome-numbers.org Dr. Rania El Sharkawy: raniashark@yahoo.com
- Define risk METHODS CONCLUSIONS ABSTRACT
- Two university laboratories and two private labor atories in Egypt submitted risk drivers QC samples run on Roche and Beckman Coulter instrumentation. The hospital laboratories selected total allowable error limits based on biological variation; the private laboratories chose CLIA limits.
- Risk drivers were entered into proprietary softwa re that applies the M.O.R.E. Quality method.
- We examined only current risk based on the measur ed mean and SD of each QC sample in the one month period relative to the package insert mean (estimated true value). Effectiveness of the existing QC Process to detect unexpected failure was beyond the scope of this study.
- We compared performance to various allowable erro r limits.
- We chose option 2 above.
- We support recommendations from the Milan confe rence , CLIA, and CLSI EP 23-A that allowable analytical error limits be based on local medical need .
- We concluded that the calculated risk level is generally not accepted in healthcare .
- Mathematically-OptimiZed Risk Evaluation detected laboratory risks that are missed by commonly used statistical QC. REFERENCES
- Clinical & Laboratory Standards Institute. Guidel ine EP23-A Laboratory quality control based on risk management. Wayne, PA. 2011
- Milan Consensus International Federation of Clini cal Chemistry and Laboratory Medicine
- Brooks Z. M.O.R.E. Quality: Lev 1. Risk Assesso r. Worthington, ON: AWEsome Numbers Inc.; 2016
- Evaluate risk: process of comparing the estimated risk against given risk criteria to determine the acceptability of the risk (ISO 14971 RESULTS RESULTS Number and cost of results that fail Biological Variation and CLIA Limits -at a simulated patient volume of 300 patents/day. (Most study labs reported fewer.) DISCUSSION: Cast Your Vote Yes No We are faced with two options:
- Laboratory errors are causing harm to a staggering number pf people and adding millions of dollars or pounds to healthcare costs, or
- Neither CLIA or Biological Variation goals can be chosen as a single source to define medical results that are ‘relevant, accurate, and reliable for patient care’ (CLIA. ) Level 2 Level 1 Red line = failed lab-selected TEa limit What do you consider acceptable quality/risk at routine/monthly QC review ? 5% allowable error 2 sigma 3 sigma 1 error/Year
MUR considered acceptable, by quality standard (Viewed by lifting panel at poster)
Patient samples / Day Total Patient samples / Year 5% allowable error 2 sigma 3 sigma 1 error/Year 5.0% 2.275% 0.135% Auto-calc’d 100 36,000 1,800 819 49 1 300 108,000 5,400 2,457 146 1 500 180,000 9,000 4,095 243 1 1,000 360,000 18,000 8,190 486 1 2. Set Acceptable Risk Criteria Cast Your Vote: METHODS -25% 0% 25%
- Recalculation showed 6 QC samples failed.