PEPconnect

General Laboratory: Total Allowable Error Online Training

Define Total Allowable Error and identify ways it can be determined. State how Total Allowable Error can be used to design an effective and efficient QC protocol for your laboratory.  This clinical laboratory training qualifies for continuing education units (CEU).

Welcome to the General Laboratory: Total Allowable Error Online Training course.  What is it and how to use it to optimize QC This course will focus on Total Allowable Error and how it is used to optimize Quality Control. Select Next to continue. This course was created by: Nils B. Person, Ph.D., FACB Senior Scientist Global Product Education   Upon successful completion of this course, you will be able to: Define Total Allowable Error   List three ways to establish Total Allowable Error   State how Total Allowable Error can be used to optimize QC procedures   Select Next to continue. QC results fail a statistical rule   or   Method performance has changed enough to impact medical care   Method performance has changed enough to impact medical care   Is that always true?   Method performance has changed enough to impact medical care Quality Requirement   = QC results fail a statistical rule   Tool Error that encompasses 95% of results True Value Bias Imprecision 5% Total Analytical Error 1.65 SD Total Analytical Error = Bias + 1.65(SD)   Total Analytical Error -the actual error that the method has Total Allowable Error -the maximum error that can be tolerated Together can be used to select optimal QC protocol   Total Allowable Error (TEa): Maximum error that can be tolerated before some outcome is affected   Not method performance based   Based on how results are used, not generated   Determined by change in outcome   Outcomes: Failed PT Altered medical decision Altered patient care Established at decision points   Examples: Glucose Na+ PSA TSH - 126 mg/dl - 115 mmol/L - 4.0 ng/dl - 4.0 µU/ml Recommended Hierarchy for Specifications:   1 - Clinical outcome studies 2 - Clinical expert opinion 3 - Biologic variation 4 - Professional recommendations 5 - Regulatory requirements 6 - State of the art 1999 Stockholm Conference: Kenny D, Fraser CG, Hyltoft Petersen P, Kallner A. Strategies to set global analytical quality specifications in laboratory medicine. Consensus agreement. Scand J Clin Lab Invest. 1999;59:585   How much change in a result alters medical outcome? That becomes the Total Allowable Error for that analyte Clinical outcome studies: Cardiac disease – Framingham, TIMI, Women’s Health Study Diabetes – DCCT, NHANES Large, prospective, long term studies looking at clinical outcome Expert Opinion: Review institutional standardized care protocols Consult with physicians for expert opinion Clinical outcome study:   Expert opinion:   DCCT: increase of HbA1c of 1% (i.e. HbA1c result going from 7% to 8%) leads to significantly poorer outcome   Endocrinologists indicate that they view a 10% change (i.e. HbA1c result going from 8% to 7.2%) indicating significant change in patient1   Total Allowable Error = 0.7% to 1.0%   1Petersen PH, Larsen ML, Horder M. Prerequisites for the maintenance of a certain state of health by biochemical monitoring. In: Harris EK, Yasada T, eds. Maintaining a Healthy State Within the Individual. Amsterdam: Elsevier; 1987:147-158.   Outcome study data do not exist for most analytes Standardized protocols often assume all lab results equivalent; do not state performance criteria Consistent results across methods / laboratories becomes critical Physicians’ intuitive sense of significant change influenced by historical variability of lab results May be conditioned by older laboratory technology Recommended Hierarchy for Specifications:   1 - Clinical outcome studies 2 - Clinical expert opinion 3 - Biologic variation 4 - Professional recommendations 5 - Regulatory requirements 6 - State of the art CV I - Within Individual Variability CVG - Between Individual Variability CVB - Total Biologic Variability This is what determines the traditional “reference interval” Total Analytical Error Current consensus goals: Desirable maximum bias = 25% of total biologic variability = 0.25(CVB)   Desirable maximum imprecision = 50% of within individual variability = 0.5(CVI)   Total Allowable Error goal:   Total Analytical Error = bias + 1.65(CV)   TEa = 0.25(CVB) + 1.65(0.5CVI)   1999 Stockholm Conference: Kenny D, Fraser CG, Hyltoft Petersen P, Kallner A. Strategies to set global analytical quality specifications in laboratory medicine. Consensus agreement. Scand J Clin Lab Invest. 1999;59:585 Desirable Specifications for Total Error, Imprecision, and Bias, Derived from Biologic Variation Ricos C, Alvarez V, Cava F, Garcia-Lario JV, Hernandez A, Jimenez CV, Minchinela J, Perich C, Simon M. "Current databases on biologic variation: pros, cons and progress." Scand J Clin Lab Invest 1999;59:491-500. Annex I, Part I: Within-subject and between-subject CV values of analytes and Desirable Analytical Quality Specifications for imprecision, bias and total error   Analyte Biologic Variation Desirable Specification CVI (%) CVB(%) CV (%) Bias (%) TEa (%) Glucose 5.7 6.9 2.9 2.2 6.9 Na+ 0.7 1.0 0.4 0.3 0.9 PSA 18.1 72.4 9.1 18.7 33.6 TSH 19.3 19.7 9.7 6.9 22.8 Analyte Biologic Variation Desirable Specification CVI (%) CVB(%) CV (%) Bias (%) TEa (%) Glucose 5.7 6.9 2.9 2.2 6.9 Na+ 0.7 1.0 0.4 0.3 0.9 PSA 18.1 72.4 9.1 18.7 33.6 TSH 19.3 19.7 9.7 6.9 22.8 Desirable Specifications for Total Error, Imprecision, and Bias, Derived from Biologic Variation Ricos C, Alvarez V, Cava F, Garcia-Lario JV, Hernandez A, Jimenez CV, Minchinela J, Perich C, Simon M. "Current databases on biologic variation: pros, cons and progress." Scand J Clin Lab Invest 1999;59:491-500. Annex I, Part I: Within-subject and between-subject CV values of analytes and Desirable Analytical Quality Specifications for imprecision, bias and total error   No complete agreement on biologically based goals Variability data for some analytes not robust Performance of some current methods cannot meet biologic goals Recommended Hierarchy for Specifications:   1999 Stockholm Conference: Kenny D, Fraser CG, Hyltoft Petersen P, Kallner A. Strategies to set global analytical quality specifications in laboratory medicine. Consensus agreement. Scand J Clin Lab Invest. 1999;59:585 1 - Clinical outcome studies 2 - Clinical expert opinion 3 - Biologic variation 4 - Professional recommendations 5 - Regulatory requirements 6 - State of the art Professional group recommendations Medical decision cutoffs and associated performance requirements TEa Goals based on Professional Recommendations: Cholesterol – NCEP - +/- 20% @ 200 mg/dl Troponin – ACC - +/- 20% @ 99th percentile TSH – NACB - +/- 19% @ 4.0 mIU/L Challenges: Published guidelines only cover limited number of analytes Standardized guidelines require consistency across methods / labs Some current methods cannot meet desired performance goals Recommended Hierarchy for Specifications:   1 - Clinical outcome studies 2 - Clinical expert opinion 3 - Biologic variation 4 - Professional recommendations 5 - Regulatory requirements 6 - State of the art National and state regulatory agencies have established acceptable limits for EQA/PT performance   1999 Stockholm Conference: Kenny D, Fraser CG, Hyltoft Petersen P, Kallner A. Strategies to set global analytical quality specifications in laboratory medicine. Consensus agreement. Scand J Clin Lab Invest. 1999;59:585 CLIA ’88 performance goals for proficiency testing Often used as examples in literature and software Goals created by committee consensus based on 1980’s technology Less than half the typical laboratory menu of analytes has CLIA PT goals Useful resource – not a gold standard   CLIA mandated PT acceptable limits Glucose Target value ± 6 mg/dl or ± 10% (greater) Sodium Target value ± 4 mmol/L PSA None Established TSH Target value ± 3SD Total Allowable Error based on CLIA PT limits Glucose 126 mg/dl ± 10% Sodium 115 mmol/L ± 3.47% PSA None TSH 4.0 µIU/ml ± 21% Challenges: Acceptable limits not defined for all analytes While limits may be based on clinical requirements, may be altered to meet practical needs of PT/EQA programs Limits must incorporate allowances for factors such as sample stability, capabilities of older technology, matrix interactions 1999 Stockholm Conference: Kenny D, Fraser CG, Hyltoft Petersen P, Kallner A. Strategies to set global analytical quality specifications in laboratory medicine. Consensus agreement. Scand J Clin Lab Invest. 1999;59:585 Recommended Hierarchy for Specifications:   1 - Clinical outcome studies 2 - Clinical expert opinion 3 - Biologic variation 4 - Professional recommendations 5 - Regulatory requirements 6 - State of the art Determining TEa is not simple   Need to use reasoned judgement   TEa can be used with different methods   No one approach can “do it all” There is no simple single formula to set TEa Tables in software and literature are examples, suggestions – not standards Goals are driven by medical need, clinical input is important Select approach for each analyte Develop goals in collaboration with clinical customers Validate goals against analytical capability – is the goal practical? Established TEa values can be used to help select methods Clinically based TEa values remain consistent unless clinical need changes Compare TEa to current method performance TEa sets the clinical error limit Method performance determines when change becomes significant If typical method error is close to Total Allowable Error, it will be very difficult to control assay performance to prevent exceeding the TEa   If typical method error is much less than TEa, it will be relatively easy to detect change in the assay’s performance before exceeding the TEa   The ratio of the method’s typical error relative to the Total Allowable Error goal has been called the Sigma Metric   How much change in the analytical process can be tolerated   Bias Imprecision True Value 1JO Westgard, Six Sigma Quality Design & Control, 2nd Ed., Westgard QC, 2006   Total Allowable Error Sigma Metric (σ): The difference between bias and TEa expressed as multiples of the SD. Sigma Metric (σ) = TEa% - Bias %                                          CV                 Total Allowable Error True Value Simple single rule QC will reliably detect method change before TEa is reached   More complex multi-rule QC protocols may be needed   Total Allowable Error True Value     Sigma Metric QC Rules 3 4 5 6 13s/R4s/22s/41s/8x            n=6   13s/R4s/22s/41s            n=4   12.5s   n=4   12.5s   n=2   13s  n=2   13.5s  n=2   JO Westgard, Six Sigma Quality Design & Control, 2nd Ed., Westgard QC Inc., 2006   Total Allowable Error σ metric will be different for each method: Does that mean different QC rules for each analyte?!   Method σ Method σ Glucose 4.8 CK 9.5 Creatinine 7.5 CEA 4.0 BUN 3.3 Cortisol 6.2 K+ 5.0 Estradiol 3.4 Na+ 2.9 Folate 6.9 Calcium 4.5 Microalbumin 9.2 LD 6.2 PSA 6.1 +/- 3 SD n=2   +/- 2.5 SD n=2   +/- 2.5 SD n=4   Multi-rule n=4   Creatinine LD CK Cortisol Folate Microalbumin Glucose K+ Calcium CEA BUN Na+ Estradiol How does this work? Currently supported in software /one time configuration   Could test one QC panel of 2 levels for all; 2nd panel for 5 methods   At the bench, no difference ⇒ QC is run – did rule fail?   Sigma Metric (σ) = TEa % - Bias %                                         CV Sigma Metric (σ) = TEa % - Bias %                                         CV Sigma Metric (σ) = TEa % - Bias %                                         CV Sigma Metric (σ) = TEa % - Bias %                                         CV To calculate need: CV: easily obtained from QC data. Be sure to use enough data over enough time to accurately reflect method TEa: already discussed challenges with determining TEa Bias: how to determine?  Bias compared to what? PT (EQA) all method mean – in many cases it’s only a peer group mean Result from “Reference Lab” – not usually reference method Reference method result for same sample(s) – best by far, but who has access to these methods?   Accuracy based surveys, Commutable Frozen Serum One pragmatic approach:  Assume bias is zero QC or PT (EQA) Peer group mean – often used, but this is most common result, not necessarily most accurate Analyte Bias CV Medical TEa CLIA TEa Biologic TEa Glucose 1% 2.3%   10% 6.9% Na+ 0% 1.0%   3.47% 0.9% PSA N/A 5.0%   None 33.6% TSH 1% 4.9% 19% 21% 22.8% Analyte Bias CV CLIA TEa Biologic TEa CLIA σ Biologic σ Glucose 1% 2.3% 10% 6.9% 3.9 2.6 Na+ 0% 1.0% 3.47% 0.9% 3.5 0.8 PSA N/A 5.0% None 33.6% None 6.7 TSH 1% 4.9% 21% 22.8% 4.1 4.4 Analyte Bias CV Medical TEa CLIA TEa Biologic TEa Glucose 1% 2.3%   10% 6.9% Na+ 0% 1.0%   3.47% 0.9% PSA N/A 5.0%   None 33.6% TSH 1% 4.9% 19% 21% 22.8% Analyte Bias CV CLIA TEa Biologic TEa CLIA σ Biologic σ Glucose 1% 2.3% 10% 6.9% 3.9 2.6 Na+ 0% 1.0% 3.47% 0.9% 3.5 0.8 PSA N/A 5.0% None 33.6% None 6.7 TSH 1% 4.9% 21% 22.8% 4.1 4.4 Analyte Bias CV CLIA TEa Biologic TEa CLIA σ Biologic σ Glucose 1% 2.3% 10% 6.9% 3.9 2.6 Na+ 0% 1.0% 3.47% 0.9% 3.5 0.8 PSA N/A 5.0% None 33.6% None 6.7 TSH 1% 4.9% 21% 22.8% 4.1 4.4 Sigma Metric (σ) = TEa % – Bias %                                          CV   TSH Medical σ = 19 % – 1 % = 3.7 (at 4.0mIU/L)                   4.9%   TSH CLIA σ = 21 % – 1 % = 4.1 (at 4.0 mIU/L)       4.9%   TSH Biologic σ = 22.8% – 1 % = 4.4 (at 4.0 mIU/L)               4.9%   Challenges: For some, no method in routine use has performance to meet biologically based goal For others, no medical or CLIA based performance goals are available There is no simple uniform way to set goals +/- 3 SD n=2   +/- 2.5 SD n=2   +/- 2.5 SD n=4   Creatinine LD CK Cortisol Folate Microalbumin PSA Glucose K+ Calcium CEA Multi-rule n=4 BUN+ NA+ Estradiol QC becomes: Simpler More effective More efficient More cost effective +/- 3 SD n=2   +/- 2.5 SD n=2   +/- 2.5 SD n=4   Multi-rule n=4   Creatinine LD CK Cortisol Folate Microalbumin PSA Glucose K+ Calcium CEA BUN Na+ Estradiol If QC results fail the rule(s): If apparent change puts results near TEa limit – hold results, act now If apparent change is still well within TEa – can still report while investigating Note: TEa should NOT be used as QC limit for rules Statistical QC detects change in performance TEa allows the change to be put in context to determine appropriate follow up Alternative to warning rules Total Allowable Error Method shift 1 - Decide on the quality goal   What is the Total Allowable Error? 2 - Evaluate method performance compared to goals What is the Sigma Metric? 3 - Choose the QC rule or rules and frequency   Use Sigma Metric to help with rule selection Best protocols will not be effective if the targets are not correctly set 4 - Set and maintain effective QC targets