PEPconnect

PEPFAR Quality Control and Method Validation Activity 3: Presentation Slides

This PDF contains the presentation slides used during the videos.

+3s +2s +1s Mean 12, rule .................... -1s violation -2s -3s 23 4 5 6 7 8 9 10 12S Rule: Data must be closely evaluated when a single control result exceeds the mean ± 2s Match the QC rule violations ACTIVITY: Match It Up 1 SD obs MEAN obs We did expect this point We did NOT expect this point(s) Y‐axis X‐axis 2 +3s +2s ( Control Limit) +1s ......................... Mean -1s 135 rule -2s violation -3s (Control rule) 1 2 3 4 5 6 7 8 9 10 (Analytical run) 3 1 HILS1752 Effective Date: 12/07/2016 Analytical Run Balancing between stability and susceptibility to change Susceptibility Introducing a change Stability 1) Known events –planned activities such as calibration, reagent changes, Maximum length –frequency of and maintenance calibration specified by the manufacturer where accuracy and 2) Unknown events – not planned, precision remains stable must determine not only what has changed but also when the Post ‐storage stability of the change occurred as part of your analyte if retesting is required recovery efforts 4 ISO 15189 5.6.2.2 Quality control materials The laboratory shall use quality control materials that react to the examining system in a manner as close as possible to patient samples. Quality control materials shall be periodically examined with a frequency that is based on the stability of the procedure and the risk of harm to the patient from an erroneous result. 5 ISO 15189:2012 ‐ 5.6.2.3 Quality control data The laboratory shall have a procedure to prevent the release of patient results in the event of quality control failure. When the quality control rules are violated and indicate that examination results are likely to contain clinically significant errors, the results shall be rejected and relevant patient samples re ‐examined after the err o r condition has been corrected and within ‐ specification performance is verified. The laboratory shall also evaluate the results from patient samples that were examined after the last successful quality control event. 6 2 HILS1752 Effective Date: 12/07/2016 8.10 Quality Control Data Comment 3 Are QC results monitored and YP N reviewed(including biases and Levy ‐ Jennings charts for quantitative tests)? Tick for each item as Yes (Y), Partial (P) or No(N) YP N a) Is there documentation of corrective action taken when quality control results exceed the acceptable range or reviews identify non conformities in a timely manner? b) Does the laboratory evaluate the results from the patient samples that were examined after the last successful quality control event? World Health Organization - -. Africa ISO15189:2012 Clause 5.6.2.3 Stepwise Laboratory Quality Improvement Process Towards Accreditation [SLIPTA) Checidist Version 2:2015 7 7 nieal atal Publle Hiraith Lab Common QC Terms  Control Rule ‐ a decision criteria to assess whether an analytical run is:  acceptable (in‐control)  not acceptable (out‐of‐control) .  Types of Control Rules  Single‐rule  uses a single criterion or a single set of control limits  13s or 12s .  Multirule  uses a combination of decision criteria with each single rule separated by a slash  13s/22s/R4s or 13s /2 of 32s /R4s /31s/6 X 8 Common QC Terms  Analytical Run ‐ the interval ( time or a group of samples) for which a decision on acceptability (in ‐control or out ‐of ‐control) is made.  Control Limit ‐ the QC chart’s defined limits or ranges expected due to random variation of the data points ‐‐ beyond those limits some course of action involving investigation and troubleshooting are taken. 9 3 HILS1752 Effective Date: 12/07/2016 Control Rule Nomenclature A A:L or A = number of control measurements involved L control limits often expressed as = the mean ± a multiple of the SD 10 Activity: Match It Up– Part I Rule Violation Purpose What will you do? • To match charts and histograms Working in groups, you will: to the correct QC rule violation.  Cut apart the rule violation nomenclature to create 9 headings. What will you need?  Match up the charts with the • Rule Violations Packet (one set appropriate heading. Use Worksheet per group) to determine how many charts match to the violation. • Worksheet: Rule Violation Answer Sheet  Attach both heading and charts to Handout 1: Bell Curve with the wall board. • Distributions  Populate first 2 columns of • Handout 2: L‐J Chart with Worksheet: Part I with the chart #. Distributions  Refer to Handouts 1 & 2 to assist • Tape your group with the matching. • Scissors 40 minutes 11 99.7% 95% 68% 2.5% 13.5% 34% 34% 13.5% 2.5% -4SD -3SD -2SD -1SD 1SD 2SD 3SD 4SD 12 4 HILS1752 Effective Date: 12/07/2016 +3 SD 2.5% + 2 SD 13.5% + 1 SD 34% 99.7% 95% 68% X 34% 1 SD - 13.5% 2 SD - 2.5% -3 SD DAY or RUN 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 13 +3 SD + 2 SD + 1 SD X 1 SD - 2 SD - -3 SD DAY or RUN 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 14 4 consecutive 4 consecutive data points data points 4SD -3SD -2SD -1SD X 1SD 2SD 3SD 4SD 15 5 HILS1752 Effective Date: 12/07/2016 Rule Violation Answer Sheet Part I Part II (SE or RE) 1:2s or 12s ____ ____ ___ 2:2s or 22s ____ ____ ___ 1:3s or 13s ____ ____ ___ 4:1s or 41s ____ ____ ___ 2 of 3:2s or 2 of 32s ____ ____ ___ 3:1s or 31s ____ ____ ___ 10x ____ ____ ___ 7T ____ ____ ___ R:4s or R4s ____ ____ ____ ___ 16 Activity: Match It Up– Part I Rule Violation Purpose What will you do? • To match charts and histograms Working in groups, you will: to the correct QC rule violation.  Cut apart the rule violation nomenclature to create 9 headings. What will you need?  Match up the charts with the • Rule Violations Packet (one set appropriate heading. Use Worksheet per group) to determine how many charts match to the violation. • Worksheet: Rule Violation Answer Sheet  Attach both heading and charts to • Handout 1: Bell Curve with the wall board. Distributions  Populate first 2 columns of • Handout 2: L‐J Chart with Worksheet: Part I with the chart #. Distributions  Refer to Handouts 1 & 2 to assist • Tape your group with the matching. • Scissors 40 minutes 17 Rule Violation Answer Sheet Part I Part II (SE or RE) 1:2s or 12s 7 9 ___ 2:2s or 22s 10 14 ___ 1:3s or 13s 1 11 ___ 4:1s or 41s 8 12 ___ 2 of 3:2s or 2 of 32s 4 16 ___ 3:1s or 31s 5 18 ___ 10x 2 13 ___ 7T 6 19 ___ R:4s or R4s 3 15 17 ___ 18 6 HILS1752 Effective Date: 12/07/2016 WHERE WE ARE  Mean ( x )  the average of a set of values  primary indicator of accuracy  measure of systematic error (error in a given direction)  Standard deviation (SD)  used to measure dispersion/scattering of a group of values around a mean  primary indicator of precision  measure of random error ( error in any direction) 19 A new population of data points are emerging due to a SMALL change in accuracy Original population when system was stable = New population as system undergoes a change = 20 Choice of Quality Control Rules Control rules should be carefully selected to maximize error detection and minimize false rejection  Error Detection – a rule violation occurs when the system is undergoing a change  False Rejection – the system is acceptable, but a QC result is outside of acceptable control limits due to chance alone  Aim for a error detection rate (Ped) ≥ 90% and a false rejection rate (Pfr) ≤5% 21 7 HILS1752 Effective Date: 12/07/2016 A new population of data points are emerging due to a LARGER change in accuracy 22 A new population of data points are emerging due to an even LARGER change in accuracy 23 A new population of data points are emerging due to a change in precision .... . Original population when system was stable = New population as system undergoes a change = 24 8 HILS1752 Effective Date: 12/07/2016 Systematic Error (change)  Consistent change in the analytical system  Error in a given direction  A change in accuracy  A shift in the observed mean value  A change in bias  Rules that look for consecutive control measurements exceeding the same control limit 25 Random Error (change)  Inconsistent change in the analytical system  Error in any direction  A change in precision  A change in SD  Rules that look at the tails of the distribution 26 Rule Violation Answer Sheet Part I Part II (SE or RE) 1:2s or 12s 7 9 ___ 2:2s or 22s 10 14 ___ 1:3s or 13s 1 11 ___ 4:1s or 41s 8 12 ___ 2 of 3:2s or 2 of 32s 4 16 ___ 3:1s or 31s 5 18 ___ 10x 2 13 ___ 7T 6 19 ___ R:4s or R4s 3 15 17 ___ 27 9 HILS1752 Effective Date: 12/07/2016 Activity: Match It Up– Part II Error Identification Purpose What will you do? • To identify whether the rule  Write the error type (SE or RE) for each violation indicates a systematic or random error. violation on a notecard.  Attach notecards next to the violation on What will you need? the wall  Wall postings from Part I of this Populate the far right column of • activity Worksheet: Part II with the type of error • Worksheet 2: Rule Violation  Each group will be assigned 1‐ 2 violations Answer Sheet to review with the class. The review must • Handout 1: Bell Curve with include: Distributions  • Handout 2: L‐J Chart with Description of the violation Distribution  Type of error • 9 Notecards  Possible cause for the error – be specific • Marker  Related Checklist item(s) • Tape 15 minutes 28 Troubleshooting Out-of-Control Runs • 1st What Rule has been violated? • 2nd Is it a Systematic or Random Error? • 3rd What is the Potential Cause of the error? • 4th If more than one analyte is out‐of‐control, do they have any Common Factors during testing? • 5th Did we Change something recently? • 6th Once we resolved the problem, did we Document the problem and its resolution? 29 “The goal of quality control is to detect, evaluate, and correct errors due to test system failure, environmental conditions, or operator performance, before patient results are reported.” LQMS Training Toolkit 30 10 HILS1752 Effective Date: 12/07/2016 8.9 Quality Control Comments 2 Is internal quality control performed, documented, and verified for all YPN tests/procedures before releasing patient results? ISO15189:2012 Clause 5.6.2 Note: QC must be verified as being within the acceptable limits before releasing results. World Health Organization www wwra Africa Stepwise Laboratory Quality Improvement Process Towards Accreditation (SLIPTA] Checklist Version 2:2015 31 For Clinical and Public Health Laboratories 11 HILS1752 Effective Date: 12/07/2016

  • match it up
  • rule violation
  • QC rules
  • measurement procedure
  • QC rule violation
  • SLIPTA
  • SLIPTA checklist
  • accuracy
  • precision
  • mean
  • SD
  • standard deviation
  • random error
  • systematic error
  • out of control run
  • LJ chart
  • histograms
  • SE
  • RE