USING TOTAL ERROR To: MONITOR PERFORMANCE, QUALIFY REAGENTS AND CALIBRATORS. AND
- SELECT Qc PROTOCOLS
REAL WORLD EXAMPLES
Zoe Brooks. George Massarella Cochrane Region Laboratory Program 700 Paris Street, Sudbury, Ont.t Canada P3E 3BS Phone Fax (705)674-8983
David Plaut Baxter Diagnostics, Miami, Florida, 33152 Phone 1-800—441-2790 Ext 6593
American Association for Clinical Chemistry
New York City July 15, 1993
INTRODUCTION
In a poster presented at AACC in 1992, we demonstrated that comparison of Total Error (T E) to a clinically-determined Total Allowable Error (TEa) is a simple and effective way to monitor changes in QC performance and to predict the acceptability of new lots of reagent and calibrators.
Here we apply this theory further in real-world examples to evaluate monthly QC, assess new lots of reagents and select appropriate QC rules.
TE will be greater than T Ea in a method with a large bias and small SD:
BIAS
TktjE
TE will be greater than T Ea in a method with a small bias and large SD:
BIAS
TRUE
• •Tk..;e.
TEa
MONTHLY QC
Routine daily QC was performed in six hospital laboratories using Dade controls in chemistry, blood gas, coagulation, special chemistry and hematology. The transmitted interlab report was imported into Paradox. Data were exported to Quattro Pro to create graphs showing the ratio of TE (Biast + 1.96 x SD) to the clinically acceptable T Ea set by G. Massarel[a for each test/control.
If TE was greater than TEa, tests were flagged and investigations performed using the TE Flowchart.
CHEMISTRY LAB X
JUNE 1993
p -05
SO 0K
so
HIGH
-2
ALB
AST 002 CREAT
1
-t
-1,5
-2
Fas UREA
so 0K
SO
BIAS OK
BIAS
PHOS L’A
The fox Ioni,r.q Tea limits indicate eLirwicaIJy a]. Iowable variation at. observed control nean values r as by G.N. HassareJLa HO r FRCPiCj valueg are modiried time to t;åae and rust: be evaluated for clinical setting.
T EST
cscot’) AST (SGOT) AZ b’,.nnin
ine Phosphatase BJ kam ine Phosphatase Bow lase Amylase el•K
ctn
cetm CEZ efil
CEL
CHI
Ca Lcium Calci.utu Chloride Ch Iorjde Cholesterol
Creatinine Creatinine oarmna-Gtr Camna—GT’ Glueose Past inq Glucose — Fascing LDEI
Phosphate Potassium Potzasgi.uffi godiurn Sodium
cnv
CHI
CH2
CHI CH2
CHI
CHI
cnv
CHL CE12
Total Bilirubin ‘l’oi:al Protein Protein t’rigLycerjdes Tr iglycerides Urate Orate Ot-ea Urea.
CHI CH2 CHI
CHI CH2 CHI
SV OHT’7’S
- a
11.0
t-i/L
2.31
JOO, 7
L 17.6
16 .04
L .06
126 . 2
1 .27 2.21. 184. ZI
5.45 1 s. 43
11/1. nmo L/ L
O
0.46 C .37
C.I’1iOI / L mmol ‘L mmö2/L
50.0
6
Li/L
38.2
o.
20
0.30 5.
nmoL/L anol/l.
nmol/L
O
640
umoI/L unxai/L
0.40 40. 0 50.0
rt’(j’10 1 / L mmol iL umol / L
rnB*ol /L
- a 141.1
56 . 9
30
g/dL g/dL
103.7
tJ/L 13/1.
:mg/dL
mEq/L
6-00
117.6
191-1
3
rnq/d:, mq/dl.
- a 36.0
nq/dL ng/dL
3.2B
6.64
112.46
7.43
0.62 L. 24
o. 5-0 4.0
2
7
o .67 4 0.84 13 7.0
nq/dL
hEq/L mF,q/I.,
mEq/L mg/dL / dL g/dL q/dL mg/dL rnq/d!. mq/dL mq/dL mc.j/dl.’ nq/dL
EXAMPLES
AMYLASE:
PROBLEM: One lab showed mean values
below other labs using the same manufacturer’s reagent. INVESTIGATION: Patient samples confirmed the bias; communication with the manufacturer indicated an incorrect reagent cal factor had been provided, RESOLUTION: Correction of the cal factor
URATE:
PROBLEM: A shift in mean value indicated
high bias. INVESTIGATION: The interlab SDI was high. Comparison of calibrator value to another lab revealed a discrepancy. The manufactureris package insert was re-checked.
RESOLUTION: A revised calibrator value
was entered.
ERROR PROBLEM-SOLVING FLOW CHART
n to C?}
TEST FLAGGED CF PRE’OIOUS 2 nepoars (7)
YES
SD/TE.Z 0.33
NO
0.20
NO
TE/TEa 1.25
YES
INVESTIGATE RANOOM EARO
INVESTIGATE SYSTEMATIC ERROR
TAKE CORRECTIVE ACTION
TAKE CORRECTIVE ACTION
CURRENT PERFORMANCE ACCEPTABLE (7)
YCS
RECORO COMMENTS tN a.c. COG
NO
MONITOR PERFORWNCE CLOSELY
ONGOING PEFCPMÅNCE ACCEPTABLE
YES
AECOAO COMMENTS IN C.C- ACTION LOG
RE-EXAMINE FACTS
IMPROVED (7)
ARRANGE TECHNO’ f ASSISTANCE
TAKE ALTERNATIVE CORRECTIVE ACTON
PATIENTS oun- UNTIL SATISFACTORY PERFORV.ANCE RE äST,4.EUSHO
NEW LOTS
Five different patient samples spanning the analytical range and two sets of control samples were tested using the trial lot of reagent or calibrator for three to five days. Anticipated changes in bias and TE were calculated.
If the new lots TE was < TEa, we accepted the lot and made appropriate changes in QC rule selection if indicated.
If the new lot’s TE was > TEa, we performed further study and rejected the lot if indicated.
ACCEPT LOT
AMYLASE Patients were unchanged; controls showed a marked shift. Pre-assessment of the lot permitted changes in QC means and avoided investigation of potential patient shift. Knowledge of the effect of the lot change on control samples explained variation in replicate proficiency samples.
ALKALINE PHOSPHATASE Means shifted SD. Results were within T Ea. Pre-assessment of the lot allowed re-assignment of QC means to avoid false QC flags.
AMYLASE FEB 1993
250
roo
150
PATIENTS
IOO 150 CURRENT LOT
300
AI-KAI-INE PHOSPHATASE
MAY 1993
goo
250
CONTROLS
PATIENTS
LOT
300 asc
REJECT LOT
PROTHROMBIN TIME Pre-assessment of the new lot showed a clinically unacceptable shift in control and patient values. Other labs confirmed the same shift in patient results. The manufacturer provided a replacement lot.
GENTAMYCIN A change in reagent composition caused a shift in controls and led to clinically-different patient results. Patient samples were referred out for comparison. This method was ultimately discontinued.
PRO-TIME NOV 1992
cow-noug
PATIENTS
15
10
CORRENT
12
GENTAMYCIN NOV 1992
CONTROLS
PATIENTS
4
2
4
8 CURRENT LOT #5TO5BUL
10
12
QC RULES
Appropriate QC rules for 2 control samples per run are selected from Westgardts QC Selection Grids based on the stability of the method and the ASEc in relation to T Ea for each control/test.
WESTGARD QC SELECTION GRID N = 2
C
A
PROCESS STABILITY (error frq•cy)
Poor Mod (2-10X) Ex (<2X)
N/A
1-2.0s
N/A
1-2.5s
1—3./2-2.z
2 to 3
1-2-os
1-2 5s
1-3 os
*SEC
3
1-2 ss
1-3 os
1-3 5s
1-30<4-10
WESTGARD Determine T Ea at medical decision limit
Determine Bias & SD
Calculate ASEc [(TEa-lBiasl)/s1-1.65
Categorize stability
Choose a QCSG; Select the row based on value of ASEc and select a column based on stability
Look up possible control rules and N
MODIFICATION Set T Ea at control means
Enter actual SD and Bias in program
Program calculates ASEc [(TEa-lBiasl)/sl-i.65
Categorize stability
Program specifies QC rules for
SAMPLE REPORT’:
f o llowi.m;j using controls
Selections per run.
are on Selection Grids cowruar. RULE SELECTION
ALB
EX
ALP ALP
BUN
EX
CAL
ceo crtn
MOD
CPR
ax
CRE CGE
Gt„tj CLU
EX
K.
Ex
CTRL YEAN
CEI
CHI
CHI
CHI
54 -GO
5.55 18. as
CH 2
CNJ
s. 06 4.09
93.10
CH2
CH 2
CBI CTt2
CHI
- go 525.70
4.37 16. OJ
EX
Ex
•rat
UAC
EX
ear
126.50
CHI. CH2
CHI
- JO
.40 .30
.10
.02
-.03
.06
- Sa
- 00
5.130 9.80
-B .50
7, so
—5 40 -25.20
.07
-.15
1.10 L. OCT
.02 .05
.02
2.30
1.00
- 20
-.80 -.40
6.40
SINGLE I HOGTI-RUI„E
1-3
90
1-3.5s
1—3 1-3 , “Gs
-lsw) 1-33/(4-
1st”
69
1-3 . os
- 03
1-3 u.
1-2. os
1-3.5s
1.09
1-2.5s
1-3
9 fi
1-3.53
2-05
1-3.03
6.91
1-3 . Ss . 59
2.65
2-3.53
3.55
1-33/(4-1gW)
N/A; trsli
I—Js/a-ag/R-4g/14-isKj
I-3s/2-2s/R—4s/4-1s
v„3G/t4-1sW) I-3s/(4-1$W)
I-3s/a-ag/R-/s/14-1sw)
1-36/ (4-1%}
I-3S/2-2s/R-4s/(4-
I—Js/f4 -mswj
EXAMPLES
For one month’s data from one lab for 24 tests, we reviewed 954 control runs, The number of runs rejected using various QC systems would have been:
Westgard Single Rule QCSG: 4 Westgard MultiRule QCSG: Full Westgard Multirules: 23 79
Some proficiency programs provide goals for internal QC. Setting SD limits in daily QC to match PT goals instead of using the lab’s actual mean and SD fails to detect
significant changes in the test system.
as avnxov a3LL07d L axvun
as nvruov yo
139HV1 -Ld CELLOId auvun
COMPARISON OF PROFICIENCY GOALS TO ACTUAL
xaouvl Ld NO
COMPARISON OF PROFICIENCY GOALS TO ACTUAL
CONCLUSIONS
1 . Comparison of TE to T Ea is a simple and effective way to monitor monthly changes in bias and imprecision.
-
Comparison of TE to T Ea when changing lots of reagent or calibrator permits appropriate changes in daily QC means, identifies unacceptable lots, and provides an indication of potential changes in proficiency testing material.
-
Clinical T Ea at control means can be applied to determine appropriate QC rules.