PEPFAR Quality Control and Method Validation Activity 5: Part 1

Analytical ' Analytical 'Analytical ; Analytical t-cequency of fest even+s Frequency of AC evenfs Y-cequcncy of evenFs Fcequency +es+ even+s t-cequency (f QC +es+ even+s Frequency (f QC even+s t-cequency of QC even+s Frequency QC even+s F cequency Of QC +es+ evenfs F—cequency Of +fes+ even+s t-cequency OF even+s + rends events avid number O type number of QC fes+ed test even esults +cends Type number of QC sampks tes+ed Iype and number o lype number o or test even number o lype and number even used 'Analytical l)) Frequency OF QC sampe +es+ even+s l) Frequency OF AC es ev Type and number sampks ies+ed of QC used +0 ased of of Frequency of QC +es+ even+s 'Analytical fest evenfs Type number of sampks ±es+ed Type and number sampks fes+ed used events Analytical Type number of (AC sampk System 2 number of QC sampks fes+ed Type number of QC sampks fes+ed procedure 2 Type and number sampks Type and number 2 Type and number of QC +es+ed 2 Type and number sampks +es+ed 2 and number Of sampks fesfed Type and number of QC sampks +es+ed Type number sampks fes+ed Type number of (r sampks Type number of (X' sampks tes+ed Type and number sampks +esfed 23A 25A •Analytical SIGNIFICJ SIGNIFIC/ 2 Type and number of sampks -tes+ed 2 Type number of QC sampks -fes+ed 2 Type and number of GC sampks -tes+ed 2 Type number of Q sampks 2 Type and number of QC sampks tes+ed 2 Type anid number of (AC sampks tes+ed , A nalyticaV Type and number of sampks Type number Of GC sampks ±es+ed SYSTEM IS S- SYSTEM IS STABLE SYSTEM IS S SYSTEM QC 2 Type and number of sampks +es+ed , Type number Of (R sampkes ±res+ed 2 Type nod number of QC sampks fes+ed Type and number Of QC sampks 2 Type number of sampks fes+ed 2 Type number Of QC sampks +e.s+ed 2 Type anod number of QC sampks ±es+ed Type and number of sampés fes+ed Type anod number Of GC sampks ±es+ed 3) QC aseA +0 QC used QC used +0 System 2) Type aud number sampks tfes+ed 2 Type number of QC sampks fes+ed 2 Type number sampks ±esfed 2 Type number of QC sampks +esfed Type numbec of r sampks tesfed QC used used +0 2 Type number sampks tes+ed Type number Of QC ±es+ed Type and number Of QC sampks tes+ed Type numbec sampks tes+ed 2 Type number Of QC sampks fes+ed Type numbec sampks +es+ed Type number Of QC sampks ±es+ed Type number sampks fesfed Type number Of GC sornpes ±es+ed Type numbec sampks ±es+ed Type number +es+ed Type number of (X sampks ±es+ed Type number sompks tes+ed SYSTEM IS STABLE QC ased +0 2) Type and number of sampks fes+ed 3) used +0 'Analytical' q) The of periodic revtew l) frequency OF QC +es+ even+s Type and number l) Frequency OF AC +es+ ev For Each QC Data Point we Need to Decide... For Each OC Data Point We Need to Decide... System 3) QC used +0 QC used +0 l) Frequency of QC +esf evenfs l) Frequency Of QC +es+ evenfs ) Frequency OF +esf even+s Frequency OF gam* +es+ even+s +0 The I frequency of QC +es+ evenfs I Frequency OF +es+ even+s I Frequency of even+s frequency of (AC +es+ evenfs I) Frequency OF +esf evenfs Frequency OF fest even+s used System l) Frequency OF +es+ events l) Frequency OF AC +es+ ever qAnaly+ical Type aod number Of sampes tes+ed Type number Of GC sampks ±es+ed 2) Type and number of GC 3) 2) Type and number Of GC samples ±es+ed Type number of ±esfed 3) QC used System l) Frequency of sampe +es+ e used QC ased +0 QC used evaluaVe QC aAsed QC I QC QC CAsed QC +0 QC used +0 QC ased + o 3) QC used QC used Frequency (f even+s QC used QC used +0 QC L(sed "Analytical QC rule QC ased +0 3) +0 3) QC 40 Label the Data Points QC +0 QC used +0 SIGNII QC QC used 40 QC ased +0 QC ase-d used used +0 2) Type and number ±es+ed 2) Type and number of GCmpks ±es+ed 2 Type number of QC sampks fes+ed WHERE WE ARE WHERE WE WHERE we ARE Type and number of GC sampks 2 Type number of sampks fes+ed Type aud number sampks fes+ed Type aud number of (X tested Type and numbec sampks tested Type and numbec of GC sampks fesfed 2) Type and number of Q sampks tes+ed Type avid number of GC sampks fes+ed 2) Type and number of QC sampks Type avid number sampks tes+ed 2) Type anid number of Q sampks fes+ed Type anid numberf of QC sampks tes+ed Type and number of QC sampks +es+ed Type number sompks hesied J Type aud number QC smaller shifts than evalua+e Analytical Quality 2) Type and number sampks 'Analytical System number 2) Type and number OfGQC sampks NO RULE 1 SD Shift 1 Shift 1 so Shift q) The of periodic review The OF periodic review For Each OC Data Point We Need to Decide- For Each OC Data Point We Heed to Decide- For Each OC Data Point We Need to Decide. shifts For Each OC Data Point We Heed to Decide For Eacn OC Data Point We Heed to Decide- For Eacn OC Data Point We Heed to Decide 3) QC used +0 •Analytical label the Data Points q) The q) The OF pevicd.c q) The Anatytical Ana tytweal IAnalytical Ana tytical Analytical Analy+ical i Analytical Ana atytical Analtytical System control limits are set q) The of periodic review q) The of peviodiC review for Ana q) The review q) The periodic review The 09 periodic review for label the Data Pmnts label the Data Pmints q) The _of periodic review __LO The review evalua+e 3) QC used 3) QC used System 3) used +0 evalua+e WHERE WE ARE Quality Quality Distribution Quality The Frequency OF periodic revtew q) The of peniodtc review QC used QC QC used Quality QC +0 Quality QC QC used +0 Quality 1 so 1 SO Shift 2 SO 1 SO Goal VIOLATION Type and number Change The QC used System periodic review for q) The periodic review used +0 used -3SD NO RULE 2 SD Shift q) The 09 pericd.c review System yste iystem The of periodic review q) The NO RULE q) The of periodic review for q) The of peviodlC revtew q) The of peviod•c review for Quality q) The peviodjc review Quality q) Thne of periodic review Quality q) The Fretuency of periodic review NO RULE used The of pevtod,c review QC used +0 q) The _oFperiodlC review Laborator Quality -2SD Analytical Analytic al Ana al Analtytical The 09 peviodlC tevtew for - 2SD q) The review The revtew The 09 petiodic review q) The of review The of periodjc review The of pev•iod.c. review The review -2SD -2SD Quality The pevic,d'C revtew Ana The of period'C review The of pevtodtc review The of vevtew for q) The of petiodtc review The Fretuency of periodic review The of pevtodvc review Laboratory The OF periodic review for The of review for The of periodic review for The of review q) The of periodic review The Fretuenc.y OF pertoåc review The penodlC revtew -2SD + 2SD The of pevicdjc review Results Laborator La boratory Laboratory The OF periodic review Laborator Goal Goals Laboratory Gnalq q) The of periodic review q) The OF periodic review 1 so Shift Gnal Goal Gnal The ofperiodjc review The of review q) The of periodic rev,e Gna Gnal *rends ts borator yrator Laborator Laboratory q) The of rev System rator -ator rato VIOLATION Laborator Quality 2 SO +cends +2SD 2 SD 2 SD Shift 2 SO Shift evalua+e System iystem Analytical Goa Goal Go VIOLATION SYSTEM IS STABLE SYSTEM IS UNSTABLE - q) The period,c review GO Goa Laborator evalua+e Control Laboratory hen resul+s +cends Goal Laborator q) The OF periodic review for Control +rends +rends Control Goals SYSTEM IS UNSTABLE - *rends +rends Anatytical ical Goal *rend S Results Goa q) The OF periodic revtew q) The of revtew q) The OF petiod•c review *rends Quality +cends Results q) The of periodic. review q) The periodic review The OF periodic revtew Results +cends Results Quality +rend s Laborato Quality Results Laborator rends q) The wency periodic review Results Results Goal Quality The periodic review ual Results Laborator q) The of perioc ages ayes q) The OF review The Fre review The revtew Its The of review The 09 periodic review for The OF periodic review The review hen resul+s ts False Accept False Accept - Results is +cends *Tend S Quality System System *rends *rend s +rends *rends Limit Results Results *rends + rends +cendS *Tends + rends Limit Laborator +rends +cend S + rends +rends +cends Laboratory Laborator The of revtew hen resul+s Results True Accept — is stable and nn QC True Accept - system is stable and no QC Laboratory I ahorator True Accept — system is stable and no QC True Accept — systemn is stable and no QC True Accept — is stable and no QC True Accept — is and QC True Accept — system is stable anci no QC True Accept — if, stable and no QC True Accept — system is and no QC True Accept Quality Gna V The periodic revtew True Accept — system is *hen Results rate Laboratory hen resul+s -ISD hen resql+s Laborator Quality Goal QC rule violati0' QC vjolati0' QC rule violati()' rule violatio en resul+s n res.d+s Laborator QC rule QC used rule QC Quality n resu14s hen resu14s hen resuH hen resu14<_- Goa hen cesul+s hen +hen +rends en resu14s ality Results +rends + rends *rends The of periodic Tevtew hen resul+s La Results hen resul+s rule violations occurred — report patients rule violations occurred — report patientG rule violations occurred - report patientG rule violations occ rule vtolations occurred — report patient% Laboratory rule violations occurred - hen resal+s hen resul+s •rends +cends hen resul+s rule vtolations rule violation occ *rend s Where hen cesul+s +hen hen resul+s rule False Acct False hen resql+s Where n resul+s hen System Mean hen Laboratory Laborator Results Results Quality Results Quality *Tend S True True Accept — system is stable True Accept — True Accept — is stable True Accept — system is Control Limit - the Results True Accept — system is stable and no QC +rend S R orator Laborator Laboratory uality hen As -3SD QC rule vio hen Results hen resul+s hen cesql+s hen *hen hen resul+s hen resd+s hen resul+s Results Quality -2SD Results Accuracy hen -2SD rule violations rule violations occurred — report patent; rule viciaticns — report patients Laborator rule violations occurred — report patients hen resul+ ua It Results we *rends Results Precision ua It Quality Quality a it it Quality ualit detections Laborator ua It Goal RULE SAMPLE ua ua 1 Com Single Results + 2SD ua It r ro we expecr EXPECT ro exPeCr ro Control HERE We ro ro ua it NO RULE ua ua 1 ast- on trot :ontrol EXPECT ro WE Results HERE we EXPECT ro ' HERE EXPECT ro BE RE VIOLATION .gso on trol Control on trot :ontrol ontrol Control, n trol wave WHERE WE ARE ua 1 on Population WHERE ua we expecr ro VERE WE ro we we exprcr ro wr ro ua ity Control WC WHERE WE ARE HERE WE TO WE EXPECT ro WE EXPECr ro WE EXPECT ro DE' ua It Control ERE WE EXPECT ro WE EXPECT ro WC EXPECr BE ; Control rejections HERE WE EXPECT ' ua Control RULE WHERE wr WHERE we WHERE ARE WHERE A Re WHERE Re QC WE we ARE WHERE ua It RULE RULE on trot on trol Control Control ontrol on tro Controt ua 1 WHERE WHERE we ARE OCCURRED WHERE WE ARE WHERE WE WHERE expected due to random WHERE WHERE we ARE WHERE WHERE ARE WHERE ARE WHERE we WHERE we ARE WHERE WHERE WE WHERE A RE iystem HERe we expecr ro WE ARE we ro WHERE ARE WHERE e expecr ro ACTIVITY we we expecr expecr ro System on trol — primary indicator of accuracy — primary indicator of precision — primary indicator of acecuracy WHERE We ua Control ua it analytical sy Analytical Control detections VIOLATION Control WHERE WE WHERE we ARE WHERE WHERE WE ARE Control VIOLATION WHERE WE WHERE WHERE WHERE W WHERE wr accuracy WHERE we ARE WHERE we WHERE we VIOLATION System WHERE WE ARE WHERE WE ua WHERE WHERE we + ISD - 2SD + 2SD Control ,ontrol ua 1 *vste ystey Control ua It yste System ON L-J ON wyeRE we WHERE WE on trol System WHERE we Re ARE .aso System System It Begins with the Right Chart Control OCCURRED OCCURRED OCCURRED + ISD +2SD -2SD +2SD -2SD PROFICIENCY Control + 2SD +2SD Control Control :ontrol CHART WHERE System PROFICIENCY WHERE WE ARE WHERE we yste PATIENT WHERE PROF-ICIENCY ON L-J yst, ON L-J ON L-J SAMPLE :ontrol WHERE we SAMPLE RULE TESTING PROFICIENCY PATIENT Control measure of systematic error (error in a given False Reject — system is stable and a QC False Reject— system is stable and a QC True Accept — system is stable and no QC False Reject- system is stable and a QC Truse Reject — systemn is nstable and ao QC Truse Accept — system is stable and no QC False Reject— system is stable anci a QC False Reject — system is stable and QC False Reject — system is stable and ao QC Control, Control SAMPLE ERE WE ' True Reject — True Accept — PROFICIENCY *OF-ICIENCY *ROFICIENCY *ROF-ICIENCY PROF-ICIENCY System yste SAMPLE CHART CHART PROFICIENCY PATIENT PROFICIENCY rule Violation occurred due to Inherent rule violations occurred — report petients rule violation occurred dure to Inherents rule Violation occurred due to inherent rule violations occurred - report patients rule violations occurred — report patients rule violation occurred d e to inherent clustering rule violation occurred due to inherent ESTING (PT) STING (PT) ESTING (PTY TESTING (PT) rule violation Oct: + 2SD + ISD rule violation occ rule violatio violation rule violation PATIENT ignore PROFICIENCY OROFICIENCY Limit Limit SAMPLE = Original population when system was st Original p = Original po = Original population when system False Reject— syste direction) = Original population when system was stable = Original False Reject— system is stable and a QC Truse Accept — system is stable and no QC True Accept — system is, stable a,d no QC False Rejecv— system is Stah False Reject— system is stable and n QC n when system was st False Reject— PROFICIENCY rejections PROFICIENCY randomness- waste of time. effort. and SS randomness. waste of time. effort, and randomness. waste of time. effort. and PATIENT randomness- waste of time, effort, and randomness- waste of time. effort, and SS randomness• waste of time. effort, and SS randomness- waste of time, randomness- waste of time. effort, and True Rejec True Reje PATIENT IICIENCY PATIENT Limit PATIENT PATIENT PROFICIENCY False Reject— system is stable an False Reject — system is unstable and a QC SAMPLE PATIENT And and ages b es ayes yes PATIENT *OF-ICIENCY ING (PT) TESTING (PT) +2SD +3SD PROFICIEN TING (PT) STING (PT) rule violation occurred — begon intignti rule violation occurred due to rule violation occur SAMPLE rule violation occurred due to inherent rule violations occ.urred —urel rule violations occurred due rule violations occurred — rel rule violation occurred dueport patent rule violations occurred durel rule violations occurred dure rule TESTING (PT) TESTING CPT) TING (PT) RULE False Accept — system is unstable and no QC Truse Accept — system is unstable and no QC rule violatio TESTING (PT) rule vio TESTING (PT) SAMPLE PROFICIENCY = New population as system undergoes a change = New population as system undergoes a PATIENT ulation as system undergoe RULE ING (PT) SAMPLE AMPLE RULE SAMPLE PATIENT SAMPLE SAMPLE PROFICIENCY RULE PATIENT SAMPLE randomness- waste of time, effort, anci randomness- waste of time, effort, anci $$ randomness- waste of time, effort. randomness- waste of time. e randomness- waste of time. effort randomness- waste Of time, rtes RULE PROFICIENCY ayes ampes QC rule violations occurred —wrong results rule violations occurred — rel SAMPLE ayes TESTING (PT) TESTING (PT) alts ages SAMPLE ayes -yes ayes ages mges ages rejections SAMPLE PATIENT PROFICIENCY SAMPLE PATIENT SAMPLE rule violations occurred — report patients rule violations occurred — re PATIENT RULE ayes b es Standard deviation (SO) Son and - 2SD TESTING (PT) PROFICIENCY clustering action involving PATIENT false SAMPLE false SAMPLE SAMPLE SAMPLE ayes ages -3SD +3SD -4SD descri b es ayes TESTING (PT) Son Slon an ISton an an Son Son Son Slon Son on It Begins With the Right Chart an escri b es ayes b es SAMPLE escft b es escrib es descfl b es iu escri b es descfl b es escribe es escribes b es b es escr. b es scf( b es descfl b es b es escfl b es b es b es ampes — used to measure dispersion/scattering of a - 2SD an descrt b e' escribe descri b and descfl b ef clustering escfl b es b es ayes b es escri es RULE SAMPLE troubleshooting are clustering on + ISD -ISD +3SD IS and he descri b es and b es descft b es and and Run 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 rand and VIOLATION and and and and and b es and ) *hen IS clustering analytical system, then we must look to our IS fhe we IS IS IS IS the -hen OCCURRED is 'IS IS IS IS IS — primary indicator of precision +cends +hen hen IS as and ON L-J is IS clustering analyiical analy+ical Analytical lyiical 'IS it is is ) +hen +hen is is is ne+ IS IS is no+ is IS is is b es IS measure of random error ( error in any occuvced. occur ced. IS away from CHART and as ) +hen +hen +rends +hends nalyiical *hen ) *hen ) Then ) +hen +hen ) +hen is direction) rejections occuvced. occurred. is ed. Inqlyiical mnaly+ical analyiical and analytica naly+ica Analytical naly+ical nalyiical naly+ical the mean False Reject— system is stable and a QC occutted neo laqel = Original population when system was stable Analytical analy+ical ealy+ica\ False Reject — system is unstable and a QC Truse Accept — system is unstable and no QC Analytical occutted. 'ccutted. Analytical analyiical qnalyiical analy+ical Analytical curved. occutced. occutted. ) +hen occurted occurced. occutfed. ed. occuvced. rule violation occurred due to inherent occuned. IS fhe rule violation occurred — begin investigation occutted. occuvced. occurted = New population as system undergoes a change occuvted. occutted. used occutted 10 IAnalytical naly± randomness- waste of time, effort, and $$ randomness- waste of time. e analy+ical and troubleshooting