Page 1

Examining healthcare cost of lab error as a risk management metric AWEsome Zoe Brooks Poster # 126 AACC 2022 Tuesday July 26, 2022 numbers

Background: This poster demonstrates a new process that brings CLSI EP23A defines riskas the”combinationof the probabilityand severity risk management as described in CLSI EP 23A to life. of harm.The number of “incorrect results-with“a failure of measurement that Updating sigma metrics to the number and cost of exceeds a limit based on medical utility” drives the probability of harm.The severity error per year enables meaningful communication of harmfrom lab erroris reflected by the healthcare costs from unnecessary follow-up between laboratory management and the C-suite and procedures ordelayed orincorrect diagnosis or treatmentreflect.Therelationship demonstrates the value of quality laboratory services. betweensize and costoferrorwasestablishedin2004forcalciumtestsby examining 89,000 Mayo Clinic charts to healthcare expenses incurred compared 1.Abstract to initial calcium tests. 2.ThisNISTstudyproved thatthehealthcarecost ofpatientcalciumresultsriseswiththevalue Method: reportedbecauseoftheincreasedfollow-uptests The authorexamined 357routine QC samples collected forvarious posters available andproceduresordered. 3.CatalystQCsoftwareextrapolatesonthat knowledgetoallowuserstospecifythehealthcare of errorwas calculatedbyCatalystQCsoftwarefromAWEsomeNumbersInc. costatfailureoftheallowableerrorlimit. Isethealthcarecostforcalcium thatfailedthe Results: CLIA limit of+/-1.0 mg/dL(0.25mmol/L) at $100 18%failed an acceptable riskstandard of 3sigma and had a cost of error above$30. (comparabletotheir2004costof$89witherror 69%passed the3sigma standardandhada cost oferrorbelowS30. of0.5mg/dL(0.125 mmol/L.) 11%passed the3sigma standard andhada cost oferrorgreater than$30. 4.Justfollowthedefinitions!Estimateriskasthe 2% of QC sampleshad a sigma above6with a cost of error above$30. PROBABILITY(number)ofMedicallyUnreliable Results(MURs) per year&SEVERITYof harm as healthcare cost. Conclusion: 5.Candidate metricsincludesigma metrics,average This studyreveals that there isno significant correlation between sigma and healthcare cost perpatient result,and the number healthcare cost oferror. ofMURsperyear.(Patientresultswereset at 100/day/QCsample) 6.Cost of error is proportional to the distribution ofdatabetween theTEalimits.Resultsclose totrue addzerohealthcarecostwhileresultsat “Based on analysis of over 89,000 patients TEa cost$1oo/each. receiving serum calcium tests at the Mayo Clinic 7.Manyresultshavecostabove$30withsigma in1998-1999,wefind that thenumber of follow-up procedures,and hence health care costs,is directlyrelated valueswell above3.0.Somehavesigmabelow toinitialcalciumtestvalues.” 3.0,appearing bad,while average cost is<$30. “For an analytical biasof0.5mg/dL,.,thepotential health care 8. With 100 patients per day, the vast majority of costincreaserangedfrom$34to$89perpatienthaving a methods produce less than one MUR/year while calciumtest…thepotentialeconomicimpactsrangefrom a 3-sigma standard allows 49. The sigma $60 million to $199 million per year histogram (#5) mask the fact that only a few methods produce >1 MUR/yr. 9.Softwarealgorithmsconsistentlyidentifymethods Cost of error increaseswith size of error thatarecloseto,orfail,acceptablerisklevels. $100 Costof avoidable Cost of avoidable $100 (Acceptableriskwassetat49MUR/yeartomirror a3sigmastandard,withacceptablecostvarying follow uptests followup tests andtreatment and treatment between$40and$50peranalyte.)Softwarethen re-analysesdatawithmodeledimprovementsto showresultantreductionsof40,oo0MURsperyear withpotentialmedicalmishapsandlitigation. 10.These 348 QC samples represent methods with Medical True Medical attainable healthcare savings of >$65 million. Goal National InstituteofStandardsandTechnology.PlanningReport Goal 04.1.Theimpact of calibration error in medical decisionmaking. (Ninesampleswereremovedfromtheoriginaldataset.) 2004.http://www.nist.gov/director/prog-ofc/report04-1.pdf

Figure 5-1.Shift in the Cost Function due to AnalyticBias Private insurance patients Projectedaverageclinicalcostoferrorperresult vsAnalyticalBias-56calciumQCsamples

000 Softwarecost $50 Study 3000 projections $40 Results $1,800 mirrorstudy $30 Reported =10.5 Cost $1,100 findings. 0.0 20 $20 True= 10.0 $650 9.5 10 10.5 11 11.5 12 50 InitialCalciumLevel -0.10 Bias

zoe@awesome-numbers.org

Page 2

Examining healthcare cost of lab error as a risk management metric AWEsome numbers

What are the best risk estimation - process used to assign values to the probability of metrics to occurrence of harm and the Severity of that harm1 estimate and evaluate the risk evaluation - process of comparing the estimated risk against given risk probability and Criteria to determine the acceptability of the risk1 severityofharm?

  1. Sigma metrics measure 2.Healthcare Cost caused by 3.Number of Medically Unreliable (thenumberofSDfromthe LabErrorperPatientResult Results(MURs)/Yr.per QCSample mean to TEa) measuresSEVERITYofharm measuresPROBABILITYofharm Sigma (TEa -|Bias|)/SD Avg Cost of LabError/Yr Medically-UnreliableResults/Yr OK Improve OK Improve

3 3to66to99to12 12to1515to18 18to21

RiskBellz RiskBellz RiskBellz Sigma =5.2, Sigma =5.2, Sigma = 2.3, 5.2s Cost = $40 23 Cost = $39 Cost = $16

Ifmethod shifts Medical True Medical Medical Trle Medical Medical True Medical to3sigma. Goal Goal Goal Goal Goal Goal $100 $50 $0 $50 S100 $100 $50 O$ $100 $100 $50 $0 $50 $100

Sigma V=<3 CostV=<$30 SigmavsNumberofMUR(Medically SigmavsCost 5,000 UnreliableResults)/Yr. A3-sigmastandardequatesto $80 OCost>=$30 4,500 Number of MUR acceptableriskof49MUR/Yr.with100 Sigma Cost v $70 Sigma<=3 4,000 MUR>49 $60 00=10000 3,500 patientsamples/day/QCsample Sigma Cost Sigma>3 3,000 $50 2,500 Sigma Cost $40 2,000 口 66%of0riginalresultsand100%of $30 1,500 Sigma β Cost $20 1,000 improvedresultsproducelessthanone 500 $10 MUR/yr. $O 1.0 2.0 3.0 1.0 2.0 3.0 4.0 9.0

OriginalDataSubmitted Softwareidentifies OPPORTUNITIES FORIMPROVEMENT Medically-Unreliable Results/Yr Avg Cost of Lab Error Improvingbias& QCsamples that are OK Improve OK Improve CV to theaverage Bias% CV% close to,or fail, OK OK ofacceptablepeers acceptableriskas preventspotential numberofcostof medical mishaps error and advises andlitigation and improvingbias or CV. reduces cost. OPPORTUNITIES FORIMPR Improve Reduce Methods Afterimprovinganalyticalprocesses Risk& Bias% Identified CV% Medically-Unreliabl Healthcare AvgCostof LabError Prevent 40K OK OK Improve OK Imp Cost OK Improve Save $65M / Yr.

MURs/Yr. with >$5 for each potential medical patient result mishaps & litigation reported

OriginalDatasubmitted After improving analytical processes QC PROCESSES PATIENTRESULTS/YR PROJECTED #MURS/YR COST OF ERROR PROJEETED #MURS/YR COST OF ERROR

348 12,699,408 40,263 $266,652,325 21 $197,768,973

zoe@awesome-numbers.org