PEPFAR Quality Control and Method Validation Activity 2: Presentation Slides

What is QC? Why is QC important to the patient? “The goal of quality control is to detect, evaluate, Essential ??? and correct errors due to test system failure, environmental conditions, or operator performance, before patient results are reported.” LQMS Training Toolkit 1 Accuracy and Precision Inaccurate but Inaccurate Precise and Imprecise SD BIAS Accurate and Precise 2 Definitions of Terms Accuracy The closeness of measurements to the true value Precision The amount of variation in the measurements Bias The difference between the expectation of a test result and an accepted reference value Reproduced from LQMS Toolkit :Quantitative QC ‐ Module 7 3 1 HILS1751 Effective Date: 12/07/2016 Gaussian (Normal) • Bell shaped curve Distribution • Symmetrical • Points cluster around the mean F • More points lie r closer to the mean e q • Farther away from u the mean, the e n fewer data points c exist y • Measures of central tendency are the Measurement same number 4 Measures of Central Tendency Mode the value which occurs with the greatest frequency Median the value at the center or midpoint of the observations Mean the calculated average of the values Gaussian Distribution: the measurement value for the mean=mode=median 5 Normal or Gaussian Bimodal Distribution Distribution Mode ≠ Median ≠ Mean Mode = Median = Mean Skewed Distribution 6 73 2 HILS1751 Effective Date: 12/07/2016 Not all central values are the same Mean Mode 12 F Median r 10 e q 8 u e n c y 2 n 0 1 2 3 4 5 6 7 8 9 10 11 12 13 Measurement Reproduced from LQMS Toolkit :Quantitative QC ‐ Module 7 7 99.7% 95% 68% When data exhibit Gaussian (Normal) distribution, we can predict the likelihood of data falling within specific percentages of the mean on the Gaussian curve. 8 Gaussian Distribution is the Key to Statistical Quality Control 9 3 HILS1751 Effective Date: 12/07/2016 O O OOOOOC What is predicted is based on past history. 10 11 99.7% 95% 68% The range of acceptable control values are calculated from the mean and standard deviation of PAST RESULTS. 12 4 HILS1751 Effective Date: 12/07/2016 Gaussian Distribution is the Key to Statistical Quality Control Gaussian Distribution, based on past results, is the predictable pattern we use to monitor a stable system. To alert us to changes (“pink circles” ) in the system, we apply control rules. 13 ISO 15189:2012 - 5.6.2.3 Quality control data Quality control data shall be reviewed at regular …………………… intervals to detect trends in examination performance that may indicate problems in the examination system. When such trends are noted, preventive actions shall be taken and recorded. NOTE Statistical and non-statistical techniques for process control should be used wherever possible to continuously monitor examination system performance. Activity: What is Statistical QC (SQC)? Purpose What will you do? By visually examining previous Work individually: data points, we can create a  Levey‐Jennings(L‐J) chart. Using Using Worksheet 2, draw the L‐J chart, we can lines and dots based upon demonstrate with the next data the facilitator’s instructions point if the system is stable or Working in groups of 4: undergoing a significant change/error.  Based upon the demonstration, create a What will you need? definition of SQC to share • Worksheet 2: Observed Mean with the class • Pencil • Ruler 45 minutes 5 HILS1751 Effective Date: 12/07/2016 M e SD obs a MEAN obs s .. u r e m e n t Y‐axis X‐axis (time of collection ) 16 Levey-Jennings Chart QC analysis relies on the ability to +3SD predict that any stable +2SD system will produce the +1SD same 99% Mean 95% distribution 68% of data on -1SD both the Gaussian -2SD curve and the QC chart. 35D To Calculate the Meanobs x xi = n Xi = individual value n = number of individual values Also known as Meancalc or Meanmeas 18 6 HILS1751 Effective Date: 12/07/2016 To Calculate the Standard Deviation (SDobs) (xi – x )2 SD = n-1 sum (of the differences) Xi = individual value Also known as X = mean of individual values n = number of individual values SDcalc or SDmeas 19 WHERE WE ARE • Mean ( x ) the average of a set of values – primary indicator of accuracy – measure of systematic error (error in a given direction) – • Standard deviation (SD) used to measure dispersion/scattering of a group of – values around a mean primary indicator of precision – measure of random error ( error in any direction) – 20 Practice: WHERE ARE WE? 130 +3 SD 120 C + 2 SD o n t 110 r + 1 SD o l 100 V X a l u 90 e 1 SD - 80 2 SD - 70 -3 SD 21 7 HILS1751 Effective Date: 12/07/2016 SD obs MEAN obs We expected this point Y‐axis X‐axis 22 99.7% 95% 68% 23 SD obs MEAN obs .. We expected this point We did NOT expect this point Y‐axis X‐axis 24 8 HILS1751 Effective Date: 12/07/2016 A new population of data points are emerging due to a change Single populations of QC data differ based on the reagent lot, calibration, and changes or errors in the analytical process 25 ISO 15189 Standard: “Performance specifications for each procedure used in an examination shall relate to the intended use of that procedure.” ISO 15189: 5.5 26 QC Workshop You need to know WHERE YOU ARE for each test you perform in your laboratory and relate it to WHERE YOU WANT TO BE for that test. 27 9 HILS1751 Effective Date: 12/07/2016 Normal or Gaussian Distribution Bimodal Distribution Mode = Median = Mean Skewed Distribution 28 99.7% 95% 68% 2.5% 13.5% 34% 34% 13.5% 2.5% -4SD -3SD -2SD -1SD 1SD 2SD 3SD 4SD XXI 29 Single Population SD obs MEAN obs We expected this point We did NOT expect this point Y‐axis Emerging Population after change X‐axis 30 10 HILS1751 Effective Date: 12/07/2016 Gaussian Distribution is the Key to Statistical Quality Control 31 On‐going Evaluation of Your Quality Control Systems Gaussian is the Key to Quality Control Alerts us to changes in Monitors and evaluates accuracy and precision method performance NS EQA TEA MN Perform parallel testing to determine the Defines an acceptable change vs. observed mean and observed SD for the unacceptable change (TE < TEA) QC material Determines how far the mean can shift Determine if the system is stable or undergoing a change before erroneous results are reported Identify rule violations when a change occurs (SEc and Sigma‐metric) Determine the type of error (SE or RE) Select appropriate control rules based on present to aid with troubleshooting the SEc or Sigma for that method 32 11 HILS1751 Effective Date: 12/07/2016