General Laboratory: Total Allowable Error Online Training
Define Total Allowable Error and identify ways it can be determined. State how Total Allowable Error can be used to design an effective and efficient QC protocol for your laboratory. This clinical laboratory training qualifies for continuing education units (CEU).
Welcome to the General Laboratory: Total Allowable Error Online Training course. What is it and how to use it to optimize QC This course will focus on Total Allowable Error and how it is used to optimize Quality Control. Select Next to continue. This course was created by: Nils B. Person, Ph.D., FACB Senior Scientist Global Product Education Upon successful completion of this course, you will be able to: Define Total Allowable Error List three ways to establish Total Allowable Error State how Total Allowable Error can be used to optimize QC procedures Select Next to continue. QC results fail a statistical rule or Method performance has changed enough to impact medical care Method performance has changed enough to impact medical care Is that always true? Method performance has changed enough to impact medical care Quality Requirement = QC results fail a statistical rule Tool Error that encompasses 95% of results True Value Bias Imprecision 5% Total Analytical Error 1.65 SD Total Analytical Error = Bias + 1.65(SD) Total Analytical Error -the actual error that the method has Total Allowable Error -the maximum error that can be tolerated Together can be used to select optimal QC protocol Total Allowable Error (TEa): Maximum error that can be tolerated before some outcome is affected Not method performance based Based on how results are used, not generated Determined by change in outcome Outcomes: Failed PT Altered medical decision Altered patient care Established at decision points Examples: Glucose Na+ PSA TSH - 126 mg/dl - 115 mmol/L - 4.0 ng/dl - 4.0 µU/ml Recommended Hierarchy for Specifications: 1 - Clinical outcome studies 2 - Clinical expert opinion 3 - Biologic variation 4 - Professional recommendations 5 - Regulatory requirements 6 - State of the art 1999 Stockholm Conference: Kenny D, Fraser CG, Hyltoft Petersen P, Kallner A. Strategies to set global analytical quality specifications in laboratory medicine. Consensus agreement. Scand J Clin Lab Invest. 1999;59:585 How much change in a result alters medical outcome? That becomes the Total Allowable Error for that analyte Clinical outcome studies: Cardiac disease – Framingham, TIMI, Women’s Health Study Diabetes – DCCT, NHANES Large, prospective, long term studies looking at clinical outcome Expert Opinion: Review institutional standardized care protocols Consult with physicians for expert opinion Clinical outcome study: Expert opinion: DCCT: increase of HbA1c of 1% (i.e. HbA1c result going from 7% to 8%) leads to significantly poorer outcome Endocrinologists indicate that they view a 10% change (i.e. HbA1c result going from 8% to 7.2%) indicating significant change in patient1 Total Allowable Error = 0.7% to 1.0% 1Petersen PH, Larsen ML, Horder M. Prerequisites for the maintenance of a certain state of health by biochemical monitoring. In: Harris EK, Yasada T, eds. Maintaining a Healthy State Within the Individual. Amsterdam: Elsevier; 1987:147-158. Outcome study data do not exist for most analytes Standardized protocols often assume all lab results equivalent; do not state performance criteria Consistent results across methods / laboratories becomes critical Physicians’ intuitive sense of significant change influenced by historical variability of lab results May be conditioned by older laboratory technology Recommended Hierarchy for Specifications: 1 - Clinical outcome studies 2 - Clinical expert opinion 3 - Biologic variation 4 - Professional recommendations 5 - Regulatory requirements 6 - State of the art CV I - Within Individual Variability CVG - Between Individual Variability CVB - Total Biologic Variability This is what determines the traditional “reference interval” Total Analytical Error Current consensus goals: Desirable maximum bias = 25% of total biologic variability = 0.25(CVB) Desirable maximum imprecision = 50% of within individual variability = 0.5(CVI) Total Allowable Error goal: Total Analytical Error = bias + 1.65(CV) TEa = 0.25(CVB) + 1.65(0.5CVI) 1999 Stockholm Conference: Kenny D, Fraser CG, Hyltoft Petersen P, Kallner A. Strategies to set global analytical quality specifications in laboratory medicine. Consensus agreement. Scand J Clin Lab Invest. 1999;59:585 Desirable Specifications for Total Error, Imprecision, and Bias, Derived from Biologic Variation Ricos C, Alvarez V, Cava F, Garcia-Lario JV, Hernandez A, Jimenez CV, Minchinela J, Perich C, Simon M. "Current databases on biologic variation: pros, cons and progress." Scand J Clin Lab Invest 1999;59:491-500. Annex I, Part I: Within-subject and between-subject CV values of analytes and Desirable Analytical Quality Specifications for imprecision, bias and total error Analyte Biologic Variation Desirable Specification CVI (%) CVB(%) CV (%) Bias (%) TEa (%) Glucose 5.7 6.9 2.9 2.2 6.9 Na+ 0.7 1.0 0.4 0.3 0.9 PSA 18.1 72.4 9.1 18.7 33.6 TSH 19.3 19.7 9.7 6.9 22.8 Analyte Biologic Variation Desirable Specification CVI (%) CVB(%) CV (%) Bias (%) TEa (%) Glucose 5.7 6.9 2.9 2.2 6.9 Na+ 0.7 1.0 0.4 0.3 0.9 PSA 18.1 72.4 9.1 18.7 33.6 TSH 19.3 19.7 9.7 6.9 22.8 Desirable Specifications for Total Error, Imprecision, and Bias, Derived from Biologic Variation Ricos C, Alvarez V, Cava F, Garcia-Lario JV, Hernandez A, Jimenez CV, Minchinela J, Perich C, Simon M. "Current databases on biologic variation: pros, cons and progress." Scand J Clin Lab Invest 1999;59:491-500. Annex I, Part I: Within-subject and between-subject CV values of analytes and Desirable Analytical Quality Specifications for imprecision, bias and total error No complete agreement on biologically based goals Variability data for some analytes not robust Performance of some current methods cannot meet biologic goals Recommended Hierarchy for Specifications: 1999 Stockholm Conference: Kenny D, Fraser CG, Hyltoft Petersen P, Kallner A. Strategies to set global analytical quality specifications in laboratory medicine. Consensus agreement. Scand J Clin Lab Invest. 1999;59:585 1 - Clinical outcome studies 2 - Clinical expert opinion 3 - Biologic variation 4 - Professional recommendations 5 - Regulatory requirements 6 - State of the art Professional group recommendations Medical decision cutoffs and associated performance requirements TEa Goals based on Professional Recommendations: Cholesterol – NCEP - +/- 20% @ 200 mg/dl Troponin – ACC - +/- 20% @ 99th percentile TSH – NACB - +/- 19% @ 4.0 mIU/L Challenges: Published guidelines only cover limited number of analytes Standardized guidelines require consistency across methods / labs Some current methods cannot meet desired performance goals Recommended Hierarchy for Specifications: 1 - Clinical outcome studies 2 - Clinical expert opinion 3 - Biologic variation 4 - Professional recommendations 5 - Regulatory requirements 6 - State of the art National and state regulatory agencies have established acceptable limits for EQA/PT performance 1999 Stockholm Conference: Kenny D, Fraser CG, Hyltoft Petersen P, Kallner A. Strategies to set global analytical quality specifications in laboratory medicine. Consensus agreement. Scand J Clin Lab Invest. 1999;59:585 CLIA ’88 performance goals for proficiency testing Often used as examples in literature and software Goals created by committee consensus based on 1980’s technology Less than half the typical laboratory menu of analytes has CLIA PT goals Useful resource – not a gold standard CLIA mandated PT acceptable limits Glucose Target value ± 6 mg/dl or ± 10% (greater) Sodium Target value ± 4 mmol/L PSA None Established TSH Target value ± 3SD Total Allowable Error based on CLIA PT limits Glucose 126 mg/dl ± 10% Sodium 115 mmol/L ± 3.47% PSA None TSH 4.0 µIU/ml ± 21% Challenges: Acceptable limits not defined for all analytes While limits may be based on clinical requirements, may be altered to meet practical needs of PT/EQA programs Limits must incorporate allowances for factors such as sample stability, capabilities of older technology, matrix interactions 1999 Stockholm Conference: Kenny D, Fraser CG, Hyltoft Petersen P, Kallner A. Strategies to set global analytical quality specifications in laboratory medicine. Consensus agreement. Scand J Clin Lab Invest. 1999;59:585 Recommended Hierarchy for Specifications: 1 - Clinical outcome studies 2 - Clinical expert opinion 3 - Biologic variation 4 - Professional recommendations 5 - Regulatory requirements 6 - State of the art Determining TEa is not simple Need to use reasoned judgement TEa can be used with different methods No one approach can “do it all” There is no simple single formula to set TEa Tables in software and literature are examples, suggestions – not standards Goals are driven by medical need, clinical input is important Select approach for each analyte Develop goals in collaboration with clinical customers Validate goals against analytical capability – is the goal practical? Established TEa values can be used to help select methods Clinically based TEa values remain consistent unless clinical need changes Compare TEa to current method performance TEa sets the clinical error limit Method performance determines when change becomes significant If typical method error is close to Total Allowable Error, it will be very difficult to control assay performance to prevent exceeding the TEa If typical method error is much less than TEa, it will be relatively easy to detect change in the assay’s performance before exceeding the TEa The ratio of the method’s typical error relative to the Total Allowable Error goal has been called the Sigma Metric How much change in the analytical process can be tolerated Bias Imprecision True Value 1JO Westgard, Six Sigma Quality Design & Control, 2nd Ed., Westgard QC, 2006 Total Allowable Error Sigma Metric (σ): The difference between bias and TEa expressed as multiples of the SD. Sigma Metric (σ) = TEa% - Bias % CV Total Allowable Error True Value Simple single rule QC will reliably detect method change before TEa is reached More complex multi-rule QC protocols may be needed Total Allowable Error True Value Sigma Metric QC Rules 3 4 5 6 13s/R4s/22s/41s/8x n=6 13s/R4s/22s/41s n=4 12.5s n=4 12.5s n=2 13s n=2 13.5s n=2 JO Westgard, Six Sigma Quality Design & Control, 2nd Ed., Westgard QC Inc., 2006 Total Allowable Error σ metric will be different for each method: Does that mean different QC rules for each analyte?! Method σ Method σ Glucose 4.8 CK 9.5 Creatinine 7.5 CEA 4.0 BUN 3.3 Cortisol 6.2 K+ 5.0 Estradiol 3.4 Na+ 2.9 Folate 6.9 Calcium 4.5 Microalbumin 9.2 LD 6.2 PSA 6.1 +/- 3 SD n=2 +/- 2.5 SD n=2 +/- 2.5 SD n=4 Multi-rule n=4 Creatinine LD CK Cortisol Folate Microalbumin Glucose K+ Calcium CEA BUN Na+ Estradiol How does this work? Currently supported in software /one time configuration Could test one QC panel of 2 levels for all; 2nd panel for 5 methods At the bench, no difference ⇒ QC is run – did rule fail? Sigma Metric (σ) = TEa % - Bias % CV Sigma Metric (σ) = TEa % - Bias % CV Sigma Metric (σ) = TEa % - Bias % CV Sigma Metric (σ) = TEa % - Bias % CV To calculate need: CV: easily obtained from QC data. Be sure to use enough data over enough time to accurately reflect method TEa: already discussed challenges with determining TEa Bias: how to determine? Bias compared to what? PT (EQA) all method mean – in many cases it’s only a peer group mean Result from “Reference Lab” – not usually reference method Reference method result for same sample(s) – best by far, but who has access to these methods? Accuracy based surveys, Commutable Frozen Serum One pragmatic approach: Assume bias is zero QC or PT (EQA) Peer group mean – often used, but this is most common result, not necessarily most accurate Analyte Bias CV Medical TEa CLIA TEa Biologic TEa Glucose 1% 2.3% 10% 6.9% Na+ 0% 1.0% 3.47% 0.9% PSA N/A 5.0% None 33.6% TSH 1% 4.9% 19% 21% 22.8% Analyte Bias CV CLIA TEa Biologic TEa CLIA σ Biologic σ Glucose 1% 2.3% 10% 6.9% 3.9 2.6 Na+ 0% 1.0% 3.47% 0.9% 3.5 0.8 PSA N/A 5.0% None 33.6% None 6.7 TSH 1% 4.9% 21% 22.8% 4.1 4.4 Analyte Bias CV Medical TEa CLIA TEa Biologic TEa Glucose 1% 2.3% 10% 6.9% Na+ 0% 1.0% 3.47% 0.9% PSA N/A 5.0% None 33.6% TSH 1% 4.9% 19% 21% 22.8% Analyte Bias CV CLIA TEa Biologic TEa CLIA σ Biologic σ Glucose 1% 2.3% 10% 6.9% 3.9 2.6 Na+ 0% 1.0% 3.47% 0.9% 3.5 0.8 PSA N/A 5.0% None 33.6% None 6.7 TSH 1% 4.9% 21% 22.8% 4.1 4.4 Analyte Bias CV CLIA TEa Biologic TEa CLIA σ Biologic σ Glucose 1% 2.3% 10% 6.9% 3.9 2.6 Na+ 0% 1.0% 3.47% 0.9% 3.5 0.8 PSA N/A 5.0% None 33.6% None 6.7 TSH 1% 4.9% 21% 22.8% 4.1 4.4 Sigma Metric (σ) = TEa % – Bias % CV TSH Medical σ = 19 % – 1 % = 3.7 (at 4.0mIU/L) 4.9% TSH CLIA σ = 21 % – 1 % = 4.1 (at 4.0 mIU/L) 4.9% TSH Biologic σ = 22.8% – 1 % = 4.4 (at 4.0 mIU/L) 4.9% Challenges: For some, no method in routine use has performance to meet biologically based goal For others, no medical or CLIA based performance goals are available There is no simple uniform way to set goals +/- 3 SD n=2 +/- 2.5 SD n=2 +/- 2.5 SD n=4 Creatinine LD CK Cortisol Folate Microalbumin PSA Glucose K+ Calcium CEA Multi-rule n=4 BUN+ NA+ Estradiol QC becomes: Simpler More effective More efficient More cost effective +/- 3 SD n=2 +/- 2.5 SD n=2 +/- 2.5 SD n=4 Multi-rule n=4 Creatinine LD CK Cortisol Folate Microalbumin PSA Glucose K+ Calcium CEA BUN Na+ Estradiol If QC results fail the rule(s): If apparent change puts results near TEa limit – hold results, act now If apparent change is still well within TEa – can still report while investigating Note: TEa should NOT be used as QC limit for rules Statistical QC detects change in performance TEa allows the change to be put in context to determine appropriate follow up Alternative to warning rules Total Allowable Error Method shift 1 - Decide on the quality goal What is the Total Allowable Error? 2 - Evaluate method performance compared to goals What is the Sigma Metric? 3 - Choose the QC rule or rules and frequency Use Sigma Metric to help with rule selection Best protocols will not be effective if the targets are not correctly set 4 - Set and maintain effective QC targets
- allowable
- error
- QC