PEPFAR Quality Control and Method Validation Activity 14 Handout: Comparison of Methods

182 Comparison of Methods Handout 14C03 Chemistry Accuracy (Trueness) Guidelines Trueness is the closeness of agreement between the average of an infinite number of replicate measured quantity values and a reference quantity value. The results of a trueness evaluation are expressed numerically as bias. Verification of trueness is the process of determining that the test system is producing accurate and valid results. Ideally, a reference measurement procedure should be used if available for the analyte being verified. If a reference measurement procedure or traceability through standard reference materials is not available, then the comparative method must be carefully selected. In the instances where the comparison method is not a reference method, then the trueness of the new method cannot be determined. The laboratory would then be measuring the difference between the methods and not the bias of the new method. Any difference between the test method and the comparative method must be carefully interpreted. I. Determine your comparison or reference method. A. The comparison method must be previously validated. B. The comparison method must be currently performing successfully in EQA. C. The ideal comparison method is a similar instrument/method. D. Comparison to an in-house method is preferred if the in-house instrument meets the above criteria. E. Samples with known values, such as proficiency testing samples or commercial standards, may be used as the reference method. II. Sample Criteria A. A minimum of 40 samples that cover the reportable range of the method and include points near the Medical Decision Points. B. Patient, quality control, and proficiency testing materials may be used. C. 50% of the selected samples must lie outside of the current reference range. III. Testing A. Run each sample in duplicate on each instrument 1. Ideally, samples should be run within 2 hours of each other unless the analyte has a shorter stability. 2. Analyze the replicates (duplicates) in different runs and in a different order. B. Retain the instrument print-outs. C. Duplicates should be averaged. D. Data should be plotted immediately to identify and correct any outliers by reviewing the Comparison Plot or Difference Plot located at the Westgard website under Paired Data Calculator. 1. Re-analyze any discrepant results between the test and comparative methods to confirm that the differences are real and not mistakes in recording the values or mix-ups of specimens. 2. If an outlier is identified, then investigate the reason and take corrective action. 3. Document the findings. 4. Remove the outlier from the data set. E. Record values on the Comparison of Methods Experiment Worksheet. HILS1746 Modified from pSMILE.org website. Equ35-A-04_Acc_Rqmt_Chem. Last accessed May 3, 2013 Effective Date: 12/6/2016 1 of 3 183 Comparison of Methods Handout 14C03 IV. Time Period of Testing A. A minimum of 5 separate days must be used for testing. B. This experiment can be performed simultaneously with the long-term precision study. V. Evaluation of Data A. Enter the data into the Paired Data Calculator located at the Westgard website. B. Calculate the slope, Y-intercept, Sy/x, and r. C. Evaluate the data using one of the options below: If Then r S 0.975 Data does not extend over acceptable range. More data must be evaluated over larger range. r >0.975 Proceed with Linear Regression Analysis to evaluate acceptability. VI. Acceptability criteria for Linear Regression Analysis A. Visually inspect the comparison plot for linearity and outliers 1. If an outlier is removed, then recalculate the regression statistics B. Visually inspect the difference plot for constant scatter throughout the AMR 1. Visually scan for significant and dramatic differences at the upper and lower ends of the range VII.Determine Bias or Difference between the Methods A. Enter Medical Decision Points (Xc) 1. A Medical Decision Point (MDP) is the concentration of the analyte at which a medical decision is triggered and/or laboratory established critical values. B. Using the linear regression equation, calculate the predicted Y value (Y') that corresponds to the concentration of Xc. C. Determine the bias (difference) by subtracting Xc from Y'. D. Calculate the % bias (% difference) as bias/Xc * 100. VIII. References A. GCLP Workshop and Workbook18-20 May 2008, Verification of Performance Specifications, pages 1-33. B. Clinical and Laboratory Standards Institute (CLSI). User Verification of Performance for Precision and Trueness: Approved Guideline-Second Edition. CLSI document EP15-A2 (ISBN 1-56238-574-7). Clinical and Laboratory Standards Institutes, 940 West Valley Road, Suite 100, Wayne, Pennsylvania 19098-1898 USA, 2005. C. Clinical and Laboratory Standards Institute (CLSI). Method Comparison and Bias Estimation Using Patient Samples: approved Guidelines- Second Edition. CLSI document EP9-A2 (ISBN 1-56238-472-4). Clinical and Laboratory Standards Institutes, 940 West Valley Road, Suite 100, Wayne, Pennsylvania 19098-1898 USA, 2005. D. Clinical and Laboratory Standards Institute (CLSI). Preliminary Evaluation of Quantitative Clinical Laboratory Measurement Procedure: Approved Guideline Third Edition. CLSI document EP10-A3 (ISBN 1-56238-622-0). ). Clinical and Laboratory Standards Institutes, 940 West Valley Road, Suite 100, Wayne, Pennsylvania 19098-1898 USA, 2005. HILS1746 Modified from pSMILE.org website. Equ35-A-04_Acc_Rqmt_Chem. Last accessed May 3, 2013 Effective Date: 12/6/2016 2 of 3 184 Comparison of Methods Handout 14C03 E. Westgard, James O. (2003). Basic Method Validation (2nd ed.). Madison: Westgard QC, Inc. F. Westgard, James O. (2008). Basic Method Validation (3rd ed.). Madison: Westgard QC, Inc. HILS1746 Modified from pSMILE.org website. Equ35-A-04_Acc_Rqmt_Chem. Last accessed May 3, 2013 Effective Date: 12/6/2016 3 of 3