PEPconnect

PEPFAR Quality Control and Method Validation Activity 11: Presentation Slides

This PDF contains the presentation slides used during the videos.

Comparing your laboratory’s performance to your peers. ACTIVITY: How Proficient Are We? 1 23 24 25 26 27 28 29 30 31 660.0 620.0 610.0 635.0 655.0 650.0 605.0 655.0 660.0 LS LS LS LS LS LS LS LS LS 23/5 24/5 25/5 26/5 27/5 28/5 29/5 30/5 31/5 MAY (current month) 2 710.0 +3 SD -------------------------- 685.0 + 2 SD C o n t 660.0 + 1 SD r o l 635.0 X V a l u 610.0 1 SD - e 585.0 2 SD - 560.0 -3 SD 23 24 25 26 27 28 29 30 31 Run 1 2 3 4 5 6 7 8 9 10 11 12 13 14 660.0 620.0 610.0 635.0 655.0 650.0 605.0 655.0 660.0 Value 610.0 615.0 635.0 660.0 590.0 635.0 680.0 655.0 660.0 610.0 625.0 LS LS LS LS LS LS LS LS LS Initials AM AM AM AM AM AM AM AM AM AM TK 23/5 24/5 25/5 26/5 27/5 28/5 29/5 30/5 31/5 Date 1/6 2/6 3/6 4/6 5/6 6/6 7/6 8/6 9/6 10/6 11/6 MAY (month ‐1) JUNE (current month) 3 1 HILS1759 Effective Date: 12/06/2016 Cape Clinic Laboratory L-J Chart for XYZ Chemistry Analyzer Analyte: Glucose Control Material: Level 1 Units: mmol/L Lot #: 3-123 Exp Date: 19/4/20XX  assign 3.48 SD assign 0.05 From: June 1, 20XX Through: June 30, 20XX Target Value 3.39 TEa 6.90% 3.63 +3 SD C 3.58 + 2 SD o n t r 3.53 + 1 SD o l V 3.48 X a l u e 3.43 1 SD - 3.38 2 SD - 3.33 -3 SD 4 Run 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 External Quality Assessment Schemes (EQAS) • Monitor the accuracy of your method on an on‐going basis • Compare your results to other laboratories performing the same method for a particular analyte 5 On‐going Evaluation of Your Quality Control Systems Gaussian is the Key to Quality Control Alerts us to changes in Monitors and evaluates accuracy and precision method performance TEa Target EQA Provides a snapshot of the current accuracy or bias of our analytical system on an ongoing basis Essential to verify the amount of bias present since bias is used in our TE and TEA equations 6 2 HILS1759 Effective Date: 12/06/2016 Types of EQAS Programs 1. Proficiency Testing (PT) Survey 2. Inter‐laboratory Comparison Program 3. Split‐sample analysis with a reference laboratory 7 Proficiency Testing (PT) Programs 1. The PT provider sends laboratories several samples from a common pool. 2. Each laboratory analyzes these samples just as they would patients. 3. Each laboratory reports its results to the PT provider 4. The PT provider evaluates the results according to specified criteria. 5. The PT provider notifies each laboratory as to the quality (proficiency) of the results in a report. 8 Hematology Sample Reported N Expected Mean SD SDI Grade Analyte/Method Result Result Platelet Count XE‐06 61 36 43 ‐ 72 57.4 2.8 1.3 Acceptable (x10E9/L) Sysmex XE‐2100 XE‐07 489 30 363 ‐ 606 484.4 11.0 0.4 Acceptable XE‐08 336 31 250 ‐ 418 333.7 8.2 0.3 Acceptable XE‐09 113 31 87 – 147 117.3 4.7 ‐0.9 Acceptable XE‐10 209 33 158 ‐ 265 211.8 7.0 ‐0.4 Acceptable 9 3 HILS1759 Effective Date: 12/06/2016 N • If N is too low, statistics may not be calculated for that peer group by the PT provider. • The higher the N, the better an estimate of the target value can be determined for that PT sample. • The higher the N, the more data points can be used to calculate the SD for the group. • The higher the N, the less impact aberrant results or incorrectly defined outliers will have on the group’s SD and/or mean. 10 Hematology Sample Reported N Expected Mean SD SDI Grade Analyte/Method Result Result Platelet Count XE‐06 61 36 43 ‐ 72 57.4 2.8 1.3 Acceptable (x10E9/L) Sysmex XE‐2100 XE‐07 489 30 363 ‐ 606 484.4 11.0 0.4 Acceptable XE‐08 336 31 250 ‐ 418 333.7 8.2 0.3 Acceptable XE‐09 113 31 87 – 147 117.3 4.7 ‐0.9 Acceptable XE‐10 209 33 158 ‐ 265 211.8 7.0 ‐0.4 Acceptable For CLIA, the %TEA for platelets is 25% TEA in units = 57.4 * 0.25 = 14.4 Lower Limit = 57.4 – 14.4 = 43 Upper limit = 57.4 + 14.4 = 71.8 rounded to 72 11 Acceptable Performance Criteria • Used to determine the limits of acceptability for the analyte • Can be determined by one of three ways 1. Target Value ± specified value 2. Percentage 3. Multiple of PT group standard deviation (SD) • If 80% of the submitted results do not fall within the limits of acceptability, then no results are graded 12 4 HILS1759 Effective Date: 12/06/2016 Hematology Sample Reported N Expected Mean SD SDI Grade Analyte/Method Result Result Platelet Count XE‐06 61 36 43 ‐ 72 57.4 2.8 1.3 Acceptable (x10E9/L) Sysmex XE‐2100 XE‐07 489 30 363 ‐ 606 484.4 11.0 0.4 Acceptable XE‐08 336 31 250 ‐ 418 333.7 8.2 0.3 Acceptable XE‐09 113 31 87 – 147 117.3 4.7 ‐0.9 Acceptable XE‐10 209 33 158 ‐ 265 211.8 7.0 ‐0.4 Acceptable 13 Standard Deviation Index (SDI) • Useful for comparing PT results at the same time • Look for systematic trends across different PT samples for a particular analyte • Serves as an “PT accuracy equalizer” to compare amount of bias with the peer group • Handy statistic to plot to trend PT results 14 For a single PT survey specimen, your laboratory submitted a creatinine result of 85 umol/L. The peer group’s mean is 75 umol/L. The peer group’s SD is 10 umol/L. What is the SDI? SDI = (your result – expected result)/group SD (85 umol/L – 75 umol/L) / 10 umol/L = 1.0 This SDI value signifies that you are 1.0 SD ABOVE the peer mean. 15 5 HILS1759 Effective Date: 12/06/2016 (85‐75)/10 = Peer 1.0 Mean This signifies that your result is 1.0 group SDs ABOVE the peer mean. 45 55 65 75 85 95 105 3SD ‐2SD ‐1SD 0SD +1SD +2SD +3SD ‐ N = 30 laboratories that submitted a data point on a single PT sample 16 Peer (90‐100)/10 Your laboratory submitted a Mean = 1.0 ‐ creatinine This signifies result of 90 that your umol/L. What result is 1.0 is the SDI? group SDs BELOW the peer mean. 70 80 90 100 110 120 130 3SD ‐2SD ‐1SD 0SD +1SD +2SD +3SD ‐ N = 30 laboratories that submitted a data point on a single PT sample 17 Survey SDI Peer Sample Mean XE‐06 1.3 XE‐07 0.4 XE‐08 0.3 XE‐09 ‐0.9 XE‐10 ‐0.4 3SD ‐2SD ‐1SD 0SD +1SD +2SD +3SD ‐ 18 6 HILS1759 Effective Date: 12/06/2016 SDI WBC Count on XE‐2100 +3 X +2 +1 XXX 0 ..... x xx x XX X X ‐1 x ‐2 ‐3 X 2012 Kit 1 2012 Kit 2 2012 Kit 3 2013 Kit 1 19 Hematology Sample Reported N Expected Mean SD SDI Grade Analyte/Method Result Result Platelet Count XE‐06 61 36 43 ‐ 72 57.4 2.8 1.3 Acceptable (x10E9/L) Sysmex XE‐2100 XE‐07 489 30 363 ‐ 606 484.4 11.0 0.4 Acceptable XE‐08 336 31 250 ‐ 418 333.7 8.2 0.3 Acceptable XE‐09 113 31 87 – 147 117.3 4.7 ‐0.9 Acceptable XE‐10 209 33 158 ‐ 265 211.8 7.0 ‐0.4 Acceptable Peer Group – comprised of other laboratories that use the same or similar instruments and methods. 20 % Lymphocyte Results for PT Specimen BCX-1 35.00 Your 34.00 33.00 32.00 31.00 Peer 30.00 Wwwwww 29.00 28.00 Group 27.00 26.00 25.00 24.00 23.00 22.00 21.00 20.00 19.00 18.00 17.00 16.00 15.00 14.00 13.00 12.00 11.00 10.00 9.00 8.00 7.00 6.00 5.00 1 2 3 4 5 21 7 HILS1759 Effective Date: 12/06/2016 Activity: Complete the PT Report Purpose What will you do? The last page of your PT report was Working in pairs, you will: smudged and unreadable. You need  Use the information supplied on to complete the assessment of the PT Worksheet to calculate the SDI report today. You decide to perform the necessary calculations to complete and Limits of Acceptability for the your analysis of free thyroxine. free thyroxine PT survey.  Compare results with the other pair working at your table. What will you need?  Determine the grade you would • Worksheet: PT Calculations receive for each PT sample. • Job Aid 2: Interpreting PT Reports  Visually plot each SDI point in the • Calculation Device (e.g. phone, histogram. calculator)  Assess the performance of free • Pencils thyroxine at your laboratory using the histogram and Job Aid 2. 22 20 minutes Directions Glaubte the SI and LimitsofArcepability. Bard on the acceptable range, determine jour pode for the survey sample. The First two rows are completed For you. OLIA acceptable performance for free thyromine ix Target Value' END of the PT group. Test Unit of Measure Limits of Acceptability Peer Group Sample Your Mean N SDI Result Lower Upper Your Grade CHIn 45 (45-464/126 464-30121 =- 05 =5.42 Sommer Advis Centa ur, XP 24 59 266-3019 26+3019 =- 14 = 209 =3.23 Somer Advis Centaur rot CHHIB 10 13 13-3008 =- 1.6 = 0.89 = 1.37 acceptable 18 BB 08-2 07y0 14 2077-3014 207+3010 =- 1.9 = 1.65 = 2.49 acceptable CHHIG 22 2 37 237-3017 237 +30.125 =- 1.4 2.01 = 2.73 acceptable = The survey ample CH-0Mix done (below)the groupsmean (agetvalue). Yourrewieis 19 group SIXfrom the perpoup'smomn. Crdp The survey ample CH-Six dove (below)the groupsmean (agetvalue) Yourreuitis 14 group SIfromthe perpop'smom Crdaon SI enables your bboratory to xxsexs your performance in the content of peer bboratories. Eventhough the performance ix satisfactory for this arolyte, your cte a problem with (X_xoraq) ( precision) foryour baboratory. chuck ni that apply ----------- 23 Peer Mean Upper Limit Lower Limit 3SD ‐2SD ‐1SD 0SD +1SD +2SD +3SD ‐ 24 8 HILS1759 Effective Date: 12/06/2016 5.6.3 Interlaboratory comparisons 5.6.3.1 Participation The laboratory shall participate in an interlaboratory comparison programme(s) (such as an external quality assessment programme or proficiency testing programme) appropriate to the examination and interpretations of examination results. The laboratory shall monitor the results of the interlaboratory comparison programme(s) and participate in the implementation of corrective actions when predetermined performance criteria are not fulfilled. ISO 15189:2012 25 9 HILS1759 Effective Date: 12/06/2016

  • proficient
  • external quality assessment schemes
  • EQAS
  • accuracy
  • standard deviation index
  • SDI
  • z-score
  • proficiency test
  • PT
  • SLIPTA
  • SLIPTA checklist
  • peer mean
  • PT failures
  • pre-analytical
  • analytical
  • post-analytical
  • CLIA
  • PT failure checklist