PEPFAR Quality Control and Method Validation Homework Assignments

QC Workshop Homework Assignments HILS1747 Effective Date: 12/16/2016 Day 1: Homework Assignments 1) Read the LQMS Training Toolkit Handout provided to you by your training facilitator. a. At the bottom of page 4 of the handout, review the steps for implementing a QC program. b. Using your SLIPTA Checklist, identify the checklist items that correspond to each program step. c. Write the checklist item number in the table. d. Complete the table and bring your responses to class. Linkage Table Steps for implementing a QC program Checklist item number(s) To establish policies and procedures 1.5, 1.8, 8.7 To assign responsibility for monitoring and reviewing To train all staff how to properly follow policies and procedures To select good QC material To establish control ranges for the selected material To develop QC graphs (L-J charts) to plot control values To establish a system for monitoring control values To take immediate corrective action if needed To maintain records of QC results and any corrective actions taken 2 HILS1747 Effective Date: 12/16/2016 Day 2: Homework Assignments 1) Complete the Evaluation Error Detection Capability Worksheet located in your homework booklet. 3 HILS1747 Effective Date: 12/16/2016 Evaluating Error Detection Capabilities Worksheet Control Material: Normal +3 SD + 2 SD + 1 SD X 1 SD - 2 SD - --- -3 SD ..... ....... ........... ................................................. ............. DAY 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 Control Material: Abnormal +3 SD + 2 SD + 1 SD X 1 SD - 2 SD - -3 SD DAY 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 4 HILS1747 Effective Date: 12/16/2016 Evaluating Error Detection Capabilities Worksheet Based on your visual examination of the L-J charts, there is a change in accuracy precision both This change occurs at analytical run # __________ . Assess the charts for rule violations. Place a tic mark in the correct column to signify a violation has occurred. For the multirule, specify which single rule(s) has been violated and how (i.e. across a run/within a material, across materials/within a run, across runs/across materials). Run # 1:2s 1:3s 1:3s/2:2s/R:4s/4:1s/8x 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 Total # of violations Multirules are harder to use than single rules. Why do we use multirules? 5 HILS1747 Effective Date: 12/16/2016 Day 3: Homework Assignment 1) Complete The Great Equalizer (%CV) Worksheet located in your homework booklet. 2) Complete the Comparison of Precision at Different Concentrations Worksheet located in your homework booklet. 3) Read Error-catch article provided to you by your training facilitator. 6 HILS1747 Effective Date: 12/16/2016 The Great Equalizer (%CV) Worksheet Your shipment of CK control materials arrived late. You only have 3-days’ worth of your current lot number before you need to use the shipment you just received to maintain your QC practices for that analyte. You check the L-J chart to see if the method’s performance has been stable. Analyte: CK Lot #: 1-123 Control Material: Level 1 Units: U/L  assign 80.0 SD assign 5.0 95.0 +3 SD C 90.0 + 2 SD o n t r 85.0 + 1 SD o l V 80.0 X a l u e 75.0 1 SD - 70.0 2 SD - 65.0 -3 SD Run 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 Analyte: CK Lot #: 3-123 Control Material: Level 3 Units: U/L  assign 635.0 SD assign 25.0 710.0 +3 SD 685.0 + 2 SD C o n t 660.0 + 1 SD r o l 635.0 X V a l u 610.0 1 SD - e 585.0 2 SD - 560.0 -3 SD Run 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 Based on your review of the provided QC charts, you conclude the following: 7 HILS1747 Effective Date: 12/16/2016 The Great Equalizer (%CV) Worksheet You recall that you only need 8-10 control measurements to determine a reasonable estimate for your observed mean for the new shipment of control materials. For the next 3 days, you analyze the new shipment in parallel with the current lot number and obtain the following measurements: Each measurement obtained on the new lot number falls within the range noted in the package insert. Level 1 (Lot # 1-456) U/L Level 3 (Lot # 3-456) U/L 67.0 611.0 Day 1 64.0 610.0 64.0 612.0 61.0 608.0 Day 2 64.0 610.0 64.0 610.0 64.0 610.0 Day 3 65.0 610.0 63.0 609.0 You know there is insufficient time to obtain a reasonable estimate of the SD for the new shipment before it must be placed into use. You also know that the SD often changes with concentration, so you are aware that you cannot use the SD directly from the current lot number and apply it to the new lot number. You decide to apply the great equalizer, %CV, because % CV can be used at the cross-over between 2 different lot numbers of control materials. Refer to Worksheet: Applying %CV 605 for assistance. You create the new L-J charts based on the new mean and calculated SD derived from the %CV. (Complete the charts.) Analyte CK Lot # 1-456 Units U/L Analyte CK Lot # 3-456 Units U/L X assigned _________ SD assigned _________ X assigned _________ SD assigned _________ +4 SD +4 SD +3 SD +3 SD C C o o n + 2 SD n + 2 SD t -------------- t r + 1 SD r + 1 SD o o -----------------.. l l X X V V a 1 SD - a 1 SD - l l -------------------- u 2 SD u 2 SD - - e -------------- e -3 SD -3 SD +4 SD 4 SD | ......... ......... RUN 1 2 3 4 5 6 …. RUN 1 2 3 4 5 6 …. 8 HILS1747 Effective Date: 12/16/2016 Comparison of Precision at Different Concentrations Worksheet Analyte: Mean True Value SD Leukocyte, Total (cell * 109 /L) (target) Low control material 2.2 2.3 0.1 Normal control material 9.4 9.6 0.6 High control material 20.5 20.0 1.1 Analyte Bias l Bias l TE in units %CV Abs Bias % TE in % (abs bias) x¯ -True Value | x¯ -True Value| Abs Bias + (1.65*SD) (SD /x¯ )*100% (Abs Bias /True Value) *100% Abs Bias % + (1.65* %CV) Low control material Normal control material High control material Illustrate the method performance of each control on the back of this worksheet. Include the target value, the Gaussian curve, an arrow representing SE (if applicable), an arrow representing RE, and an arrow representing TE. 9 HILS1747 Effective Date: 12/16/2016 Day 4: Homework Assignments Day 4: Homework Assignments (continued) 1) Complete the Diagraming Method Performance Worksheet located in your homework booklet. 2) Complete the SEc Investigation on CK located in your homework booklet. 10 HILS1747 Effective Date: 12/16/2016 Diagraming Method Performance Worksheet Directions: Given the 4 key numbers, complete the table. Using the values from the table, diagram the method’s performance. Label the following in your diagram using arrows: SE, RE, TE, SEc. Example 1 Mean 96 mmol/L SD 2 mmol/L True (target) Value 100 mmol/L %TEA 16% TEA in units Lower TEA Limit Upper TEA Limit SE (absolute bias) RE (using a z-score of 1.65) TE (SE+RE) SEc (using a z-score of 1.65) 11 HILS1747 Effective Date: 12/16/2016 Diagraming Method Performance Worksheet Example 2 Mean 102 mmol/L SD 2 mmol/L True (target) Value 100 mmol/L %TEA 12% TEA in units Lower TEA Limit Upper TEA Limit SE (absolute bias) RE (using a z-score of 1.65) TE (SE+RE) SEc (using a z-score of 1.65) Why is the SEc value for Example 2 less than the SEc for Example 1 even though Example 1’s TE is greater than Example 2’s TE? 12 HILS1747 Effective Date: 12/16/2016 SEc Investigation on CK Directions: Using the same handout, SEc Monthly Review Log, you used to investigate of calcium, glucose, potassium, and sodium, perform an investigation of CK. Complete the Case Scenario Investigation Worksheet. CK-Level 1 Cape Clinic Laboratory L-J Chart for XYZ Chemistry Analyzer Analyte: CK Control Material: Level 1 Units: U/L Lot #: 1-123 Exp Date: 19/4/20XX  assign 80.0 SD assign 5.0 From: June 1, 20XX Through: June 30, 20XX Target Value 81.0 TEa 30.30% 95.0 +3 SD C 90.0 + 2 SD o n t r 85.0 + 1 SD o l V 80.0 X a l u e 75.0 1 SD - 70.0 2 SD - 65.0 -3 SD Run 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 Value 80.0 83.0 70.0 82.0 85.0 80.0 75.0 89.0 78.0 84.0 76.0 83.0 75.0 71.0 86.0 80.0 84.0 76.0 80.0 83.0 89.0 77.0 85.0 84.0 77.0 80.0 82.0 71.0 78.0 76.0 Initials AM AM AM AM AM AM AM AM AM AM TK TK TK TK TK TK TK TK TK TK LS LS LS LS LS LS LS LS LS LS Date 1/6 2/6 3/6 4/6 5/6 6/6 7/6 8/6 9/6 10/6 11/6 12/6 13/6 14/6 15/6 16/6 17/6 18/6 19/6 20/6 21/6 22/6 23/6/ 24/6 25/6 26/6 27/6 28/6 29/6 30/6 13 HILS1747 Effective Date: 12/16/2016 SEc Investigation on CK CK-Level 3 Cape Clinic Laboratory L-J Chart for XYZ Chemistry Analyzer Analyte: CK Control Material: Level 3 Units: U/L Lot #: 3-123 Exp Date: 19/4/20XX  assign 635.0 SD assign 25.0 From: June 1, 20XX Through: June 30, 20XX Target Value 638.0 TEa 30.3% 710.0 - +3 SD 685.0 + 2 SD C o n t 660.0 + 1 SD r o l 635.0 X V a l u 610.0 1 SD - e 585.0 2 SD - 560.0 -3 SD Run 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 Value 610.0 615.0 635.0 660.0 590.0 635.0 680.0 655.0 660.0 610.0 625.0 655.0 650.0 615.0 655.0 685.0 615.0 615.0 635.0 590.0 625.0 655.0 660.0 620.0 610.0 635.0 655.0 650.0 605.0 655.0 Initials AM AM AM AM AM AM AM AM AM AM TK TK TK TK TK TK TK TK TK TK LS LS LS LS LS LS LS LS LS LS Date 1/6 2/6 3/6 4/6 5/6 6/6 7/6 8/6 9/6 10/6 11/6 12/6 13/6 14/6 15/6 16/6 17/6 18/6 19/6 20/6 21/6 22/6 23/6/ 24/6 25/6 26/6 27/6 28/6 29/6 30/6 14 HILS1747 Effective Date: 12/16/2016 SEc Investigation on CK Case Scenario Investigation Worksheet What did you observe on the QC Charts for the analyte CK during the month of June 20XX? What evidence did you collect to support your observation? Record Name Information Obtained from Record What is your assessment of the underlying cause of this issue? What recommendations would you give to this site’s QA Officer? 15 HILS1747 Effective Date: 12/16/2016 Day 5: Homework Assignments 1) Complete Find the Key Numbers Worksheet located in your homework booklet. 2) Read Getting Practical about Measurement Uncertainty article provided to you by your training facilitator. 16 HILS1747 Effective Date: 12/16/2016 Find the Key Numbers Worksheet Directions: Given the diagram, what are the 4 key numbers? Example 1 WHERE WE ARE WHERE WE WANT TO BE Mean (mmol/L) True Value (mmol/L) SD (mmol/L) %TEA TE = 11.6 RE = 1.65 SD SE = 5 TEA TEA Lower Limit Upper Limit 170 X True 230 Value 200 Example 2 WHERE WE ARE WHERE WE WANT TO BE Mean (mmol/L) True Value (mmol/L) SD (mmol/L) %TEA TE = 24.5 which includes 95% of the measurements RE = 16.5 TEA TEA Lower Limit Upper Limit 164 True X 236 Value 200 17 HILS1747 Effective Date: 12/16/2016 Day 6: Homework Assignments 1) Develop your improvement project plan for this workshop. a. Worksheet 2: Action Plan Template 1309 located on page 141 of your QC Workshop Participant’s Manual. b. Worksheet 3: IP Timeline 1310 located on page 142 of your QC Workshop Participant’s Manual. 2) Read Understanding the Meaning of Accuracy, Trueness, and Precision article provided to you by your training facilitator. 3) Read the appropriate experiment handout and accompanying worksheet from the QC Workshop Participant’s Manual for your group assignment. I am in Group # ___________ . Group 1 - Group 2 - Group 3 - Group 4 – Linearity Precision Comparison of Reference Interval Experiment Experiment Methods Experiment page 173 page 176 Experiment page 181 page 189 Handout Handout, Handout, Handout, pages 172-174 pages 177-178 pages 182-184 pages 190-193 Accompanying Accompanying Accompanying Accompanying Worksheet Worksheet Worksheet Worksheet pages 175 pages 179-180 pages 185-186 page 194 18 HILS1747 Effective Date: 12/16/2016 Day 7: Homework Assignments 1) Read Validation 101 article provided to you by your training facilitator. 19 HILS1747 Effective Date: 12/16/2016