View source for Journal:Use of middleware data to dissect and optimize hematology autoverification

{{Infobox journal article
|name         = 
|image        = 
|alt          = <!-- Alternative text for images -->
|caption      = 
|title_full   = Use of middleware data to dissect and optimize hematology autoverification
|journal      = ''Journal of Pathology Informatics''
|authors      = Starks, Rachel D.; Merrill, Anna E.; Davis, Scott R.; Voss, Dena R.; Goldsmith, Pamela, J.; Brown, Bonnie S.; Kulhavy, Jeff; Krasowski, Matthew D.
|affiliations = University of Iowa Hospitals and Clinics
|contact      = [https://www.jpathinformatics.org/login.asp?rd=article.asp?issn=2153-3539;year=2021;volume=12;issue=1;spage=19;epage=19;aulast=Starks Log-in required]
|editors      = 
|pub_year     = 2021
|vol_iss      = '''12'''
|pages        = 19
|doi          = [https://doi.org/10.4103/jpi.jpi_89_20 10.4103/jpi.jpi_89_20]
|issn         = 2153-3539
|license      = [https://creativecommons.org/licenses/by-nc-sa/4.0/ Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International]
|website      = [https://www.jpathinformatics.org/text.asp?2021/12/1/19/313145 https://www.jpathinformatics.org/text.asp?2021/12/1/19/313145]
|download     = [https://www.jpathinformatics.org/temp/JPatholInform12119-643471_175227.pdf https://www.jpathinformatics.org/temp/JPatholInform12119-643471_175227.pdf] (PDF)
}}

==Abstract==
'''Background''': [[Hematology]] [[analysis]] comprises some of the highest volume tests run in [[Clinical laboratory|clinical laboratories]]. Autoverification of hematology results using computer-based rules reduces turnaround time for many [[Sample (material)|specimens]], while strategically targeting specimen review by technologist or pathologist. 

'''Methods''': Autoverification rules had been developed over a decade at an 800-bed tertiary/quarternary care academic medical central laboratory serving both adult and pediatric populations. In the process of migrating to newer hematology instruments, we analyzed the rates of the autoverification rules/flags most commonly associated with triggering manual review. We were particularly interested in rules that on their own often led to manual review in the absence of other flags. Prior to the study, autoverification rates were 87.8% (out of 16,073 orders) for complete blood count (CBC) if ordered as a panel and 85.8% (out of 1,940 orders) for CBC components ordered individually (not as the panel). 

'''Results''': Detailed analysis of rules/flags that frequently triggered indicated that the immature granulocyte (IG) flag (an instrument parameter) and rules that reflexed platelet by impedance method (PLT-I) to platelet by fluorescent method (PLT-F) represented the two biggest opportunities to increase autoverification. The IG flag threshold had previously been validated at 2%, a setting that resulted in this flag alone preventing autoverification in 6.0% of all samples. The IG flag threshold was raised to 5% after detailed chart review; this was also the instrument vendor's default recommendation for the newer hematology analyzers. Analysis also supported switching to PLT-F for all platelet analysis. Autoverification rates increased to 93.5% (out of 91,692 orders) for CBC as a panel and 89.8% (out of 11,982 orders) for individual components after changes in rules and laboratory practice. 

'''Conclusions''': Detailed analysis of autoverification of hematology testing at an academic medical center clinical laboratory that had been using a set of autoverification rules for over a decade revealed opportunities to optimize the parameters. The [[data analysis]] was challenging and time-consuming, highlighting opportunities for improvement in software tools that allow for more rapid and routine evaluation of autoverification parameters.

'''Keywords''': algorithms, clinical laboratory information system, hematology, informatics, middleware

==Introduction==
In the realm of [[laboratory information system]] (LIS) and/or [[middleware]] software, autoverification refers to the use of computer-based rules to determine the appropriate release of [[laboratory]] test results. With the expansion of data management systems in the lab, autoverification is now a routine practice in core [[Clinical laboratory|clinical laboratories]]<ref name=":0">{{Cite journal |last=Crolla |first=Lawrence J. |last2=Westgard |first2=James O. |date=2003-09 |title=Evaluation of rule-based autoverification protocols |url=https://pubmed.ncbi.nlm.nih.gov/14531220 |journal=Clinical leadership & management review: the journal of CLMA |volume=17 |issue=5 |pages=268–272 |issn=1527-3954 |pmid=14531220}}</ref><ref name=":1">{{Cite journal |last=Jones |first=Jay B. |date=2013-03 |title=A strategic informatics approach to autoverification |url=https://pubmed.ncbi.nlm.nih.gov/23331736 |journal=Clinics in Laboratory Medicine |volume=33 |issue=1 |pages=161–181 |doi=10.1016/j.cll.2012.11.004 |issn=1557-9832 |pmid=23331736}}</ref><ref>{{Cite journal |last=Pearlman |first=Eugene S. |last2=Bilello |first2=Leonard |last3=Stauffer |first3=Joseph |last4=Kamarinos |first4=Andonios |last5=Miele |first5=Rudolph |last6=Wolfert |first6=Marc S. |date=2002-07 |title=Implications of autoverification for the clinical laboratory |url=https://pubmed.ncbi.nlm.nih.gov/12168427 |journal=Clinical leadership & management review: the journal of CLMA |volume=16 |issue=4 |pages=237–239 |issn=1527-3954 |pmid=12168427}}</ref><ref name=":2">{{Cite journal |last=Torke |first=Narayan |last2=Boral |first2=Leonard |last3=Nguyen |first3=Tracy |last4=Perri |first4=Angelo |last5=Chakrin |first5=Alan |date=2005-12 |title=Process improvement and operational efficiency through test result autoverification |url=https://pubmed.ncbi.nlm.nih.gov/16306113 |journal=Clinical Chemistry |volume=51 |issue=12 |pages=2406–2408 |doi=10.1373/clinchem.2005.054395 |issn=0009-9147 |pmid=16306113}}</ref>, where the use of well-designed autoverification rules improves both [[Quality (business)|quality]] and efficiency.<ref name=":0" /><ref name=":1" /><ref name=":2" /> Over the years, autoverification rules have been described in detail for clinical chemistry, blood gas, and coagulation analysis, often achieving autoverification rates of >90%.<ref name=":8">{{Cite journal |last=Krasowski |first=Matthew D. |last2=Davis |first2=Scott R. |last3=Drees |first3=Denny |last4=Morris |first4=Cory |last5=Kulhavy |first5=Jeff |last6=Crone |first6=Cheri |last7=Bebber |first7=Tami |last8=Clark |first8=Iwa |last9=Nelson |first9=David L. |last10=Teul |first10=Sharon |last11=Voss |first11=Dena |date=2014 |title=Autoverification in a core clinical chemistry laboratory at an academic medical center |url=https://pubmed.ncbi.nlm.nih.gov/24843824 |journal=Journal of Pathology Informatics |volume=5 |issue=1 |pages=13 |doi=10.4103/2153-3539.129450 |issn=2229-5089 |pmc=4023033 |pmid=24843824}}</ref><ref name=":11">{{Cite journal |last=Sediq |first=Amany Mohy-Eldin |last2=Abdel-Azeez |first2=Ahmad GabAllahm Hala |date=2014-09 |title=Designing an autoverification system in Zagazig University Hospitals Laboratories: preliminary evaluation on thyroid function profile |url=https://pubmed.ncbi.nlm.nih.gov/25827700 |journal=Annals of Saudi Medicine |volume=34 |issue=5 |pages=427–432 |doi=10.5144/0256-4947.2014.427 |issn=0975-4466 |pmc=6074554 |pmid=25827700}}</ref><ref name=":3">{{Cite journal |last=Onelöv |first=Liselotte |last2=Gustafsson |first2=Elisabeth |last3=Grönlund |first3=Eva |last4=Andersson |first4=Helena |last5=Hellberg |first5=Gisela |last6=Järnberg |first6=Ingela |last7=Schurow |first7=Sara |last8=Söderblom |first8=Lisbeth |last9=Antovic |first9=Jovan P. |date=2016-10 |title=Autoverification of routine coagulation assays in a multi-center laboratory |url=https://pubmed.ncbi.nlm.nih.gov/27400327 |journal=Scandinavian Journal of Clinical and Laboratory Investigation |volume=76 |issue=6 |pages=500–502 |doi=10.1080/00365513.2016.1200135 |issn=1502-7686 |pmid=27400327}}</ref><ref name=":12">{{Cite journal |last=Randell |first=Edward W. |last2=Short |first2=Garry |last3=Lee |first3=Natasha |last4=Beresford |first4=Allison |last5=Spencer |first5=Margaret |last6=Kennell |first6=Marina |last7=Moores |first7=Zoë |last8=Parry |first8=David |date=2018-06 |title=Strategy for 90% autoverification of clinical chemistry and immunoassay test results using six sigma process improvement |url=https://pubmed.ncbi.nlm.nih.gov/29904674 |journal=Data in Brief |volume=18 |pages=1740–1749 |doi=10.1016/j.dib.2018.04.080 |issn=2352-3409 |pmc=5998219 |pmid=29904674}}</ref><ref name=":13">{{Cite journal |last=Randell |first=Edward W. |last2=Short |first2=Garry |last3=Lee |first3=Natasha |last4=Beresford |first4=Allison |last5=Spencer |first5=Margaret |last6=Kennell |first6=Marina |last7=Moores |first7=Zoë |last8=Parry |first8=David |date=2018-05 |title=Autoverification process improvement by Six Sigma approach: Clinical chemistry & immunoassay |url=https://pubmed.ncbi.nlm.nih.gov/29518383 |journal=Clinical Biochemistry |volume=55 |pages=42–48 |doi=10.1016/j.clinbiochem.2018.03.002 |issn=1873-2933 |pmid=29518383}}</ref><ref name=":14">{{Cite journal |last=Wu |first=Jie |last2=Pan |first2=Meichen |last3=Ouyang |first3=Huizhen |last4=Yang |first4=Zhili |last5=Zhang |first5=Qiaoxin |last6=Cai |first6=Yingmu |date=2018-12 |title=Establishing and Evaluating Autoverification Rules with Intelligent Guidelines for Arterial Blood Gas Analysis in a Clinical Laboratory |url=https://pubmed.ncbi.nlm.nih.gov/29787327 |journal=SLAS technology |volume=23 |issue=6 |pages=631–640 |doi=10.1177/2472630318775311 |issn=2472-6311 |pmid=29787327}}</ref><ref name=":15">{{Cite journal |last=Randell |first=Edward W. |last2=Yenice |first2=Sedef |last3=Khine Wamono |first3=Aye Aye |last4=Orth |first4=Matthias |date=2019-11 |title=Autoverification of test results in the core clinical laboratory |url=https://pubmed.ncbi.nlm.nih.gov/31386832 |journal=Clinical Biochemistry |volume=73 |pages=11–25 |doi=10.1016/j.clinbiochem.2019.08.002 |issn=1873-2933 |pmid=31386832}}</ref><ref name=":16">{{Cite journal |last=Wang |first=Zhongqing |last2=Peng |first2=Cheng |last3=Kang |first3=Hui |last4=Fan |first4=Xia |last5=Mu |first5=Runqing |last6=Zhou |first6=Liping |last7=He |first7=Miao |last8=Qu |first8=Bo |date=2019-07-03 |title=Design and evaluation of a LIS-based autoverification system for coagulation assays in a core clinical laboratory |url=https://pubmed.ncbi.nlm.nih.gov/31269951 |journal=BMC medical informatics and decision making |volume=19 |issue=1 |pages=123 |doi=10.1186/s12911-019-0848-2 |issn=1472-6947 |pmc=6609390 |pmid=31269951}}</ref>

In contrast, published studies regarding the application of autoverification in hematopathology are more limited.<ref name=":17">{{Cite journal |last=Fu |first=Qiang |last2=Ye |first2=Congxiu |last3=Han |first3=Bo |last4=Zhan |first4=Xiaoxia |last5=Chen |first5=Kang |last6=Huang |first6=Fuda |last7=Miao |first7=Lisao |last8=Yang |first8=Shanhong |last9=Wang |first9=Weijia |date=2020-04-01 |title=Designing and Validating Autoverification Rules for Hematology Analysis in Sysmex XN-9000 Hematology System |url=https://pubmed.ncbi.nlm.nih.gov/32255287 |journal=Clinical Laboratory |volume=66 |issue=4 |doi=10.7754/Clin.Lab.2019.190726 |issn=1433-6510 |pmid=32255287}}</ref><ref name=":4">{{Cite journal |last=Zhao |first=X. |last2=Wang |first2=X. F. |last3=Wang |first3=J. B. |last4=Lu |first4=X. J. |last5=Zhao |first5=Y. W. |last6=Li |first6=C. B. |last7=Wang |first7=B. H. |last8=Wei |first8=J. |last9=Guo |first9=P. |last10=Xiao |first10=J. P. |last11=Wang |first11=J. H. |date=2016-04 |title=Multicenter study of autoverification methods of hematology analysis |url=https://pubmed.ncbi.nlm.nih.gov/27358150 |journal=Journal of Biological Regulators and Homeostatic Agents |volume=30 |issue=2 |pages=571–577 |issn=0393-974X |pmid=27358150}}</ref> Zhao ''et al.'' describe the implementation of autoverification rules in [[hematology]] [[analysis]] in a multicenter setting with 76%–85% autoverification rates.<ref name=":4" /> The necessity of manual review of peripheral blood smears precludes achieving the high autoverification rates seen in clinical chemistry. On the other hand, high rates of manual review may place a strain on limited laboratory resources and delay turnaround time without adding clinical value. In 2005, The International Consensus Group for Hematology (ICGH) issued guidelines to establish a uniform set of criteria for manual review of automated hematology testing.<ref name=":5">{{Cite journal |last=Buoro |first=Sabrina |last2=Mecca |first2=Tommaso |last3=Seghezzi |first3=Michela |last4=Manenti |first4=Barbara |last5=Azzarà |first5=Giovanna |last6=Ottomano |first6=Cosimo |last7=Lippi |first7=Giuseppe |date=2017-07 |title=Validation rules for blood smear revision after automated hematological testing using Mindray CAL-8000 |url=https://pubmed.ncbi.nlm.nih.gov/27709664 |journal=Journal of Clinical Laboratory Analysis |volume=31 |issue=4 |doi=10.1002/jcla.22067 |issn=1098-2825 |pmc=6817000 |pmid=27709664}}</ref><ref name=":6">{{Cite journal |last=Froom |first=Paul |last2=Havis |first2=Rosa |last3=Barak |first3=Mira |date=2009 |title=The rate of manual peripheral blood smear reviews in outpatients |url=https://pubmed.ncbi.nlm.nih.gov/19778287 |journal=Clinical Chemistry and Laboratory Medicine |volume=47 |issue=11 |pages=1401–1405 |doi=10.1515/CCLM.2009.308 |issn=1437-4331 |pmid=19778287}}</ref><ref>{{Cite journal |last=Palmer |first=L. |last2=Briggs |first2=C. |last3=McFadden |first3=S. |last4=Zini |first4=G. |last5=Burthem |first5=J. |last6=Rozenberg |first6=G. |last7=Proytcheva |first7=M. |last8=Machin |first8=S. J. |date=2015-06 |title=ICSH recommendations for the standardization of nomenclature and grading of peripheral blood cell morphological features |url=https://pubmed.ncbi.nlm.nih.gov/25728865 |journal=International Journal of Laboratory Hematology |volume=37 |issue=3 |pages=287–303 |doi=10.1111/ijlh.12327 |issn=1751-553X |pmid=25728865}}</ref><ref name=":7">{{Cite journal |last=Pratumvinit |first=Busadee |last2=Wongkrajang |first2=Preechaya |last3=Reesukumal |first3=Kanit |last4=Klinbua |first4=Cherdsak |last5=Niamjoy |first5=Patama |date=2013-03 |title=Validation and optimization of criteria for manual smear review following automated blood cell analysis in a large university hospital |url=https://pubmed.ncbi.nlm.nih.gov/23451752 |journal=Archives of Pathology & Laboratory Medicine |volume=137 |issue=3 |pages=408–414 |doi=10.5858/arpa.2011-0535-OA |issn=1543-2165 |pmid=23451752}}</ref> The proposed criteria for manual review includes quantitative and qualitative parameters. Pratumvinit ''et al.'' optimized the ICGH guidelines to significantly reduce their review rates and increase autoverification.<ref name=":7" /> The basic qualitative criteria used for manual review are well-established; however, the specific quantitative cutoffs to trigger manual review are largely set by the individual laboratory, with some recommendations for individual parameters provided by instrument vendors or published literature.<ref name=":3" /><ref name=":5" /><ref name=":6" /><ref>{{Cite journal |last=Barnes |first=P. W. |date=2005 |title=Comparison of performance characteristics between first- and third-generation hematology systems |url=https://pubmed.ncbi.nlm.nih.gov/16475477 |journal=Laboratory Hematology: Official Publication of the International Society for Laboratory Hematology |volume=11 |issue=4 |pages=298–301 |doi=10.1532/lh96.05037 |issn=1080-2924 |pmid=16475477}}</ref><ref>{{Cite journal |last=Barth |first=David |date=2012-02 |title=Approach to peripheral blood film assessment for pathologists |url=https://pubmed.ncbi.nlm.nih.gov/22372204 |journal=Seminars in Diagnostic Pathology |volume=29 |issue=1 |pages=31–48 |doi=10.1053/j.semdp.2011.07.003 |issn=0740-2570 |pmid=22372204}}</ref><ref>{{Cite journal |last=Rabizadeh |first=Esther |last2=Pickholtz |first2=Itay |last3=Barak |first3=Mira |last4=Froom |first4=Paul |date=2013-08 |title=Historical data decrease complete blood count reflex blood smear review rates without missing patients with acute leukaemia |url=https://pubmed.ncbi.nlm.nih.gov/23505267 |journal=Journal of Clinical Pathology |volume=66 |issue=8 |pages=692–694 |doi=10.1136/jclinpath-2012-201423 |issn=1472-4146 |pmid=23505267}}</ref> Individual laboratories ideally should optimize their own set of rules to maintain both quality and efficiency within their own context of instrumentation, staffing, and patient population. However, [[data analysis]] on specific flags and their clinical impact may be quite challenging to assess.

In this study, we evaluated autoverification rules at an 800-bed tertiary/quarternary academic medical center core clinical laboratory for a complete blood count (CBC) with white blood cell (WBC) count differential (Diff) and the “''a la carte''” ordering of individual CBC components. The laboratory had developed and validated autoverification protocols over a decade. Feedback from laboratory staff suggested that some rules were resulting in manual review without clear clinical benefit. We therefore sought opportunities for improvement by assessing the flags that most frequently held [[Sample (material)|specimens]] for manual review. Our analysis also illustrates some of the data analytical challenges associated with evaluating hematology autoverification.

==Methods==
===Institutional details===
The present study was performed at an approximately 800-bed tertiary/quaternary care academic medical center. The medical center services included pediatric and adult inpatient units, multiple intensive care units (ICUs), a level I trauma-capable emergency treatment center, and outpatient services. Pediatric and adult hematology/oncology services include both inpatient and outpatient populations. For the purpose of this study, patients 18 years and older were classified as adults, with pediatric patients < 18-years old. The data in the study were collected as part of a retrospective study approved by the university Institutional Review Board (protocol #201801719) covering the time period from January 1, 2018, to July 31, 2018. This study was carried out in accordance with the Code of Ethics of the World Medical Association (declaration of Helsinki).

===Data extraction and analysis===
The [[electronic health record]] (EHR) throughout the retrospective study period was Epic (Epic Systems, Inc., Madison, Wisconsin, USA), which has been in place since May 2009. The middleware software was Data Innovations (DI) Instrument Manager (DI, Burlington Vermont, USA) version 8.14, with the autoverification rules predominantly contained within the DI middleware.<ref name=":8" /><ref>{{Cite journal |last=Grieme |first=Caleb V. |last2=Voss |first2=Dena R. |last3=Davis |first3=Scott R. |last4=Krasowski |first4=Matthew D. |date=2017-03-01 |title=Impact of Endogenous and Exogenous Interferences on Clinical Chemistry Parameters Measured on Blood Gas Analyzers |url=https://pubmed.ncbi.nlm.nih.gov/28271676 |journal=Clinical Laboratory |volume=63 |issue=3 |pages=561–568 |doi=10.7754/Clin.Lab.2016.160932 |issn=1433-6510 |pmid=28271676}}</ref> The LIS was Epic Beaker Clinical Pathology.<ref name=":9">{{Cite journal |last=Krasowski |first=Matthew D. |last2=Wilford |first2=Joseph D. |last3=Howard |first3=Wanita |last4=Dane |first4=Susan K. |last5=Davis |first5=Scott R. |last6=Karandikar |first6=Nitin J. |last7=Blau |first7=John L. |last8=Ford |first8=Bradley A. |date=2016 |title=Implementation of Epic Beaker Clinical Pathology at an academic medical center |url=https://pubmed.ncbi.nlm.nih.gov/26955505 |journal=Journal of Pathology Informatics |volume=7 |pages=7 |doi=10.4103/2153-3539.175798 |issn=2229-5089 |pmc=4763507 |pmid=26955505}}</ref> Data were extracted from DI using Microsoft Open Database Connectivity (Microsoft Corporation, Redmond, Washington, USA) and analyzed using Microsoft Excel. Instrument flag data were retrieved from the analyzer and required extensive data cleanup and manual review to assure integrity. One major challenge is that the error messages concatenate on one another in a variety of combinations. Additional File 1 (see Notes at bottom) shows an example of the data, de-identified to remove identifying data fields related to accession number, dates/times, and personnel performing the testing. The flag fields are not transmitted to Epic Beaker Clinical Pathology<ref name=":9" /> nor are the operation identification numbers that specify who reviewed, released, and rejected results. These fields would be needed to calculate percent autoverification in the LIS if that were a goal.

===Instrument flags===
In our laboratory, instrument flags are generated either from the automated hematology instrument manufacturer (Sysmex, America) or by our own laboratory-validated rules built in middleware (summarized in Table 1, and indicating origin of rule). These flags are either global (i.e., applied to every sample) or patient-specific (e.g., a patient known to have previous samples that required special handling or analysis). When a sample triggers a flag, several outcomes are possible: (1) automatically release the CBC component results but hold the WBC Diff for manual review, (2) hold both the CBC and WBC Diff for manual review, and (3) release all results to the LIS/EHR without manual review (assuming no other flags intervene). For example, the flag for the presence of immature granulocytes (IG) above a set percentage will hold only the WBC Diff and release the CBC, while the thrombocytopenia flag will hold both the CBC and WBC Diff for manual review. IGs on manual review include metamyelocytes, myelocytes, and promyelocytes. Critical value flags, in the absence of other flags, do not preclude autoverification; notification of the clinical services for critical values is by telephone per protocol.


[[File:Tab1 Starks JPathInfo2021 12.jpg|1010px]]
{{clear}}
{| 
 | style="vertical-align:top;" |
{| border="0" cellpadding="5" cellspacing="0" width="1010px"
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" |<blockquote>'''Table 1.''' Flags for manual review of complete blood cell count and white blood cell count differential tests</blockquote>
 |- 
|}
|}

===Automated analyzers===
Automated hematology testing was performed by a Sysmex XN-9000 hematology analyzer with a fully automated hematology slide preparation and staining system (Sysmex America, Inc., Lincolnshire, Illinois, USA). This instrument performs platelet (PLT) enumeration either by disruption of electrical current (PLT-I) or by a flow cytometric method using a fluorescent oxazine dye (PLT-F). Briefly, for the PLT-F method, the dye binds to platelet organelles, is then irradiated by laser beam, and the corresponding forward-scattered light and side-scattered fluorescence are plotted.<ref name=":10">{{Cite journal |last=Tanaka |first=Yuzo |last2=Tanaka |first2=Yumiko |last3=Gondo |first3=Kazumi |last4=Maruki |first4=Yoshiko |last5=Kondo |first5=Tamiaki |last6=Asai |first6=Satomi |last7=Matsushita |first7=Hiromichi |last8=Miyachi |first8=Hayato |date=2014-09 |title=Performance evaluation of platelet counting by novel fluorescent dye staining in the XN-series automated hematology analyzers |url=https://pubmed.ncbi.nlm.nih.gov/24648166 |journal=Journal of Clinical Laboratory Analysis |volume=28 |issue=5 |pages=341–348 |doi=10.1002/jcla.21691 |issn=1098-2825 |pmc=6807536 |pmid=24648166}}</ref> The PLT-F method better distinguishes between platelets and fragmented red blood cells.<ref name=":10" /><ref name=":20">{{Cite journal |last=Schoorl |first=Margreet |last2=Schoorl |first2=Marianne |last3=Oomes |first3=Jeanette |last4=van Pelt |first4=Johannes |date=2013-10 |title=New fluorescent method (PLT-F) on Sysmex XN2000 hematology analyzer achieved higher accuracy in low platelet counting |url=https://pubmed.ncbi.nlm.nih.gov/24045545 |journal=American Journal of Clinical Pathology |volume=140 |issue=4 |pages=495–499 |doi=10.1309/AJCPUAGGB4URL5XO |issn=1943-7722 |pmid=24045545}}</ref><ref>{{Cite journal |last=Wada |first=Atsushi |last2=Takagi |first2=Yuri |last3=Kono |first3=Mari |last4=Morikawa |first4=Takashi |date=2015 |title=Accuracy of a New Platelet Count System (PLT-F) Depends on the Staining Property of Its Reagents |url=https://pubmed.ncbi.nlm.nih.gov/26496387 |journal=PloS One |volume=10 |issue=10 |pages=e0141311 |doi=10.1371/journal.pone.0141311 |issn=1932-6203 |pmc=4619826 |pmid=26496387}}</ref> During the timeframe for the present study, PLT-F used higher cost reagents than PLT-I (approximately 50% more at onset of project).

==Results==
===Volume of testing and frequency of flags===
Over a six-month period, a total of 132,432 specimens had CBC with or without WBC Diff or an ''a la carte'' order for individual CBC components (PLT, hemoglobin, and hematocrit). Manual review by a technologist was performed on 10,314 of those specimens (7.8%). During this period, a total of 53,396 instrument flags were triggered (note that an individual specimen may trigger up to 15 flags), with 80.3% of samples not associated with any flag. Overall, 9.7% of specimens triggered a single flag, 5.0% triggered two flags, and < 1% of samples triggered five or more flags (Figure 1a).

Pediatric ICUs (including both neonatal and pediatric units) had the highest percentage of flagged samples, with one or more flags on 52.5% of specimens (Figure 1b). Adult and pediatric non-ICU inpatient units had 29.6% and 28.4% samples, respectively, with at least one flag. Adult hematology/oncology services, which include both an inpatient bone marrow transplant unit and outpatient clinics, had a 28.8% rate of samples with one or more flags. Rate of sample flags was much lower in outpatient (excluding hematology/oncology), emergency department, and operating room locations, at approximately 10% or less in both adult and pediatric populations.


[[File:Fig1 Starks JPathInfo2021 12.jpg|500px]]
{{clear}}
{| 
 | style="vertical-align:top;" |
{| border="0" cellpadding="5" cellspacing="0" width="500px"
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" |<blockquote>'''Figure 1.''' The number of samples during a six-month period without an associated flag (80.3%) or with one to four flags are shown in ('''a'''). The distribution of samples by patient care area for adult and pediatric patients is shown in ('''b'''). Heme/Onc: Hematology/Oncology, ICU: Intensive care unit, ED: Emergency department, OR: Operating room</blockquote>
 |- 
|}
|}

===Frequently triggered flags===
To analyze the patterns of flags that frequently triggered manual review for both WBC and PLT parameters, we began by reviewing WBC parameters. This was limited to a 30-day period of analysis due to the extensive nature of data cleanup and manual review for the middleware and instrument data. We looked at two outcomes: (1) flags that would release the CBC while holding the WBC Diff for manual review and (2) flags that would hold both the CBC and WBC Diff for manual review. In the first category of releasing the CBC and holding the WBC Diff for manual review, the IG present flag represented 9.6% of flags during a 30-day review period (20,576 samples and 1,980 flags) (Figure 2a). The next most frequently triggered flag was the WBC abnormal scattergram at 5.3% (1,087 flags), followed by abnormal lymphocytes or blasts flag at 4.7% (962 flags) (Figure 2a). These top three most frequently triggered flags are instrument flags, with the ≥2% IG cutoff specified by the laboratory (discussed in more detail below).

For platelets, the PLT-I method was the main methodology used to generate a platelet count, with PLT-F used in certain circumstances. Samples were run for PLT-F based on the following flags: (1) PLT-I <70 k/mm<sup>3</sup> (“thrombocytopenia”), (2) 50% change in either direction within the last seven days (“delta failure”), (3) pediatric inpatients and pediatric hematology/oncology clinic patients (due to known higher rate of red blood cell fragmentation and other specimen challenges), and/or (4) platelet abnormal distribution flag on the hematology analyzer. For 20,576 samples and 1,637 flags during the review period, we identified PLT-I <70 k/mm<sup>3</sup> as accounting for 8.0% of flags that were holding both the CBC and WBC Diff to re-run for PLT-F (Figure 2b). The next most frequently triggered flags to hold CBC and WBC Diff for manual review were PLT clumps (2.2%, 460 flags) and PLT delta failure (1.7%, 349 flags) (Figure 2b).


[[File:Fig2 Starks JPathInfo2021 12.jpg|500px]]
{{clear}}
{| 
 | style="vertical-align:top;" |
{| border="0" cellpadding="5" cellspacing="0" width="500px"
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" |<blockquote>'''Figure 2.''' The most frequently triggered flags that resulted in manual review of WBC differential while automatically releasing the CBC during a 30-day period are shown in ('''a'''), with IG Present as the only flag triggered in 9.6% of samples. In ('''b'''), the six-most frequently triggered flags that hold both the CBC and WBC differential for manual review are shown, with the most frequently triggered flag Thombocytopenia, Rerun PLT-F, 8.0%. IG: Immature granulocytes, Abn WBCs: Abnormal white blood cells, Abn Lymphs/Blasts: Abnormal lymphocytes or blasts, Lymphs: Lymphocytes, MCV: Mean corpuscular volume, PLT: Platelet, HGB: Hemoglobin</blockquote>
 |- 
|}
|}

===Most frequently triggered single flag===
Next, we examined the samples during a six-month period that had only a single flag. By far, the IG flag (intended to detect metamyelocytes, myelocytes, and promyelocytes) was the most frequently triggered single-flag, representing 6.0% of flags (3,200 samples) (Figure 3a). The left shift and the abnormal lymphocyte/blasts flags both represent 0.80% (each 425 flags), while 0.37% of single flags (199 flags) was due to the WBC abnormal scattergram (Figure 3a). All four flags are generated by instrument rules. The left shift flag primarily detects bands and metamyelocytes. In 1.1% of samples, the IG and left shift flags occurred together and were the only flags present (608 flags) (Figure 3a).


[[File:Fig3 Starks JPathInfo2021 12.jpg|500px]]
{{clear}}
{| 
 | style="vertical-align:top;" |
{| border="0" cellpadding="5" cellspacing="0" width="500px"
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" |<blockquote>'''Figure 3.''' When a single flag for manual review was triggered, the four most frequent rules identified are shown, including a potential overlap of parameters in IG Present and Left Shift in ('''a'''). Shown in ('''b''') is the difference in manual review rates when the IG cutoff is changed from ≥2% (804 samples) to ≥5% (234 samples). IG: Immature granulocytes, Abn Lymphs or Blasts: Abnormal lymphocytes or blasts, WBC Abn Scatter: White blood cell abnormal scattergram</blockquote>
 |- 
|}
|}

===Optimization of immature granulocyte flag===
The IG flag data prompted us to perform a more detailed review of the clinical utility of this flag. The IG flag had been set for ≥2% based on a validation study performed on an earlier generation of hematology analyzer used in the laboratory. The instrument vendor recommended a default trigger for the IG flag at 5%, while a range of 3–5% IG has been reported in the literature.<ref>{{Cite journal |last=Eilertsen |first=Heidi |last2=Hagve |first2=Tor-Arne |date=2014-10 |title=Do the flags related to immature granulocytes reported by the Sysmex XE-5000 warrant a microscopic slide review? |url=https://pubmed.ncbi.nlm.nih.gov/25239424 |journal=American Journal of Clinical Pathology |volume=142 |issue=4 |pages=553–560 |doi=10.1309/AJCP4V4EXYFFOELL |issn=1943-7722 |pmid=25239424}}</ref><ref>{{Cite journal |last=Fernandes |first=Bernard |last2=Hamaguchi |first2=Yukio |date=2007-09 |title=Automated enumeration of immature granulocytes |url=https://pubmed.ncbi.nlm.nih.gov/17709320 |journal=American Journal of Clinical Pathology |volume=128 |issue=3 |pages=454–463 |doi=10.1309/TVGKD5TVB7W9HHC7 |issn=0002-9173 |pmid=17709320}}</ref><ref>{{Cite journal |last=Maenhout |first=Thomas M. |last2=Marcelis |first2=Ludo |date=2014-07 |title=Immature granulocyte count in peripheral blood by the Sysmex haematology XN series compared to microscopic differentiation |url=https://pubmed.ncbi.nlm.nih.gov/24668849 |journal=Journal of Clinical Pathology |volume=67 |issue=7 |pages=648–650 |doi=10.1136/jclinpath-2014-202223 |issn=1472-4146 |pmid=24668849}}</ref> In order to assess the effect on our patient population if we changed the IG parameter to ≥5%, we performed detailed chart review on CBC samples that had only the IG rule triggered.

In a 30-day period, 804 samples underwent manual review due solely to the IG flag with the rule set to trigger at ≥2%; of those reviewed, only 29.1% (234 samples) had an IG of ≥5% (Figure 3b, above). Of the 570 samples with <5% IG but ≥2%, most came from inpatient units, with a breakdown of 412 inpatients (72.3%), 145 outpatients (25.4%), and 13 emergency department patients (2.3%). Within the 570 samples, manual chart review identified 4.7% samples from 27 unique patients with promyelocytes (0.9–2.0%) and one with blasts (0.9%). All of these samples were from patients on inpatient or adult hematology/oncology services and were follow-up specimens from patients already worked-up and being followed for hematologic issues. Fourteen patients with promyelocytes identified were positive for malignancy, six of which were simultaneously receiving chemotherapy. Seventeen of the 27 patients identified with promyelocytes were receiving daily CBCs during an inpatient encounter. The data were then analyzed to see how the IG estimate compared to the identification of metamyelocytes, myelocytes, and promyelocytes in these specimens by a technologist. Manual review of the 570 samples led to lower %IG in 91.1% of samples and higher %IG in only 8.6% of samples. Thus, the IG flag appears to over-estimate relative to manual slide review.

Extrapolating from the one month of data, samples with <5% IG but ≥2% comprise an estimated 6,840 samples per year. Given that chart review of this subset did not identify any case where the manual review led to the identification of promyelocytes or blasts that had not already been identified in previous laboratory studies, we made a decision to raise the IG threshold to 5% to match the manufacturer recommendation. Thereafter, the IG parameter, if present as the only flag, only triggered manual review if 5% or greater. The change in this threshold did not impact measurement of other flags.

===Decreased review and re-running of complete blood counts with PLT-F===
Based on the data and support from the published literature, the laboratory made the decision to switch to the PLT-F method instead of the PLT-I method. Similar to the change in IG threshold, the switch to PLT-F method had the highest impact on inpatient samples, with a breakdown of 59.2% inpatient (15.1% of which was ICUs), 31.0% outpatient, and 9.8% emergency department samples during the period of the study. The biggest impact on autoverification resulted from not needing to perform PLT-F for PLT-I <70 k/mm<sup>3</sup>.

===Overall impact of changes===
In combination with the above-mentioned change in IG threshold, autoverification rates increased. Figure 4 compares the autoverification rates before and after the changes in PLT-F and IG threshold. The percent increase in autoverification was 5.7% for CBC as a panel and 4.0% for individual CBC components. This translates to an estimated absolute reduction in manual review of 13,266 CBC panels and 1,248 CBC individual components per year. This has substantial impact on turnaround time for individual samples, since average turnaround time for manual differential is about 90 minutes, depending on staffing levels and competing workload. Average time to actually perform manual differential depends on complexity of pathologic findings and technologist experience but is typically five to fifteen minutes. Using ten minutes as an approximate average time for review, the reduction would translate to nearly a full-time equivalent position (approximately 100 hours/year or nearly 300 eight-hour shifts).


[[File:Fig4 Starks JPathInfo2021 12.jpg|500px]]
{{clear}}
{| 
 | style="vertical-align:top;" |
{| border="0" cellpadding="5" cellspacing="0" width="500px"
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" |<blockquote>'''Figure 4.''' Comparison of platelet-related flags with the switch to universal use of platelet by fluorescent method (PLT-F) method shown in ('''a''') and autoverification rates for complete blood count and individually ordered complete blood count-components in ('''b''')</blockquote>
 |- 
|}
|}

==Discussion==
There is a growing body of literature related to the development and optimization of autoverification rules in hematopathology.<ref name=":17" /><ref name=":4" /> This complements investigations of autoverification for clinical chemistry, blood gas, and coagulation analysis.<ref name=":8" /><ref name=":11" /><ref name=":3" /><ref name=":12" /><ref name=":13" /><ref name=":14" /><ref name=":15" /><ref name=":16" /> Hematopathology presents particular challenges for autoverification in that rules are intended for a range of purposes, including review of abnormal cells that might be misidentified or missed by instruments (e.g., blasts, Sezary cells), detection of phenomena that can distort analysis (e.g., RBC agglutination and platelet clumping), and unusual changes in quantitative parameters (e.g., dramatic decrease or increase in hemoglobin/hematocrit).<ref name=":17" /><ref name=":4" /><ref>{{Cite journal |last=Lantis |first=Kay L. |last2=Harris |first2=R. Jayne |last3=Davis |first3=Gerald |last4=Renner |first4=Nancy |last5=Finn |first5=William G. |date=2003-05 |title=Elimination of instrument-driven reflex manual differential leukocyte counts. Optimization of manual blood smear review criteria in a high-volume automated hematology laboratory |url=https://pubmed.ncbi.nlm.nih.gov/12760283 |journal=American Journal of Clinical Pathology |volume=119 |issue=5 |pages=656–662 |doi=10.1309/VH1K-MV8W-B7GB-7R14 |issn=0002-9173 |pmid=12760283}}</ref> Some of the flags are associated with phenomena that might be a pre-analytical sample issue or a pathological process in the patient.<ref name=":4" /><ref name=":7" /><ref name=":18">{{Cite journal |last=Comar |first=Samuel Ricardo |last2=Malvezzi |first2=Mariester |last3=Pasquini |first3=Ricardo |date=2017-10 |title=Evaluation of criteria of manual blood smear review following automated complete blood counts in a large university hospital |url=https://pubmed.ncbi.nlm.nih.gov/29150102 |journal=Revista Brasileira De Hematologia E Hemoterapia |volume=39 |issue=4 |pages=306–317 |doi=10.1016/j.bjhh.2017.06.007 |issn=1516-8484 |pmc=5693276 |pmid=29150102}}</ref><ref>{{Cite journal |last=Ike |first=Samuel O. |last2=Nubila |first2=Thomas |last3=Ukaejiofo |first3=Ernest O. |last4=Nubila |first4=Imelda N. |last5=Shu |first5=Elvis N. |last6=Ezema |first6=Ifeyinwa |date=2010-04-23 |title=Comparison of haematological parameters determined by the Sysmex KX - 2IN automated haematology analyzer and the manual counts |url=https://pubmed.ncbi.nlm.nih.gov/20416068 |journal=BMC clinical pathology |volume=10 |pages=3 |doi=10.1186/1472-6890-10-3 |issn=1472-6890 |pmc=PMC2873444 |pmid=20416068}}</ref><ref name=":19">{{Cite journal |last=Lou |first=Amy H. |last2=Elnenaei |first2=Manal O. |last3=Sadek |first3=Irene |last4=Thompson |first4=Shauna |last5=Crocker |first5=Bryan D. |last6=Nassar |first6=Bassam A. |date=2017-10 |title=Multiple pre- and post-analytical lean approaches to the improvement of the laboratory turnaround time in a large core laboratory |url=https://pubmed.ncbi.nlm.nih.gov/28457964 |journal=Clinical Biochemistry |volume=50 |issue=15 |pages=864–869 |doi=10.1016/j.clinbiochem.2017.04.019 |issn=1873-2933 |pmid=28457964}}</ref><ref>{{Cite journal |last=Sandhaus |first=Linda M. |last2=Wald |first2=David N. |last3=Sauder |first3=Kenan J. |last4=Steele |first4=Erica L. |last5=Meyerson |first5=Howard J. |date=2007-03 |title=Measuring the clinical impact of pathologist reviews of blood and body fluid smears |url=https://pubmed.ncbi.nlm.nih.gov/17516750 |journal=Archives of Pathology & Laboratory Medicine |volume=131 |issue=3 |pages=468–472 |doi=10.5858/2007-131-468-MTCIOP |issn=1543-2165 |pmid=17516750}}</ref> A primary challenge for autoverification in hematopathology is to balance efficiency and turnaround time while performing manual review for samples, where the review is likely to provide clinical benefit.<ref name=":2" /><ref name=":4" /><ref name=":18" /><ref name=":19" /><ref>{{Cite journal |last=Novis |first=David A. |last2=Walsh |first2=Molly |last3=Wilkinson |first3=David |last4=St Louis |first4=Mary |last5=Ben-Ezra |first5=Jonathon |date=2006-05 |title=Laboratory productivity and the rate of manual peripheral blood smear review: a College of American Pathologists Q-Probes study of 95,141 complete blood count determinations performed in 263 institutions |url=https://pubmed.ncbi.nlm.nih.gov/16683868 |journal=Archives of Pathology & Laboratory Medicine |volume=130 |issue=5 |pages=596–601 |doi=10.5858/2006-130-596-LPATRO |issn=1543-2165 |pmid=16683868}}</ref> This is especially a challenge for laboratories that analyze a high percentage of samples from patients with hematologic abnormalities, especially those who undergo repeated laboratory analysis over time.

In the present study, we evaluated autoverification rules that had been developed over years in our core clinical laboratory. In this process, we were confronted with rules that had been adopted per manufacturer recommendation (especially instrument flags) and those that had been developed and validated over years into an autoverification rule set. We were particularly looking for rules and thresholds that might represent “low hanging fruit” in generating the high frequency of flags but with low clinical value.

A central challenge identified in our study is the difficulty in extracting and analyzing specific data for autoverification. Our laboratory uses middleware software for most of the autoverification rules. Data retrieval required running a third-party application every month to capture middleware data prior to off-site archival (where the extraction would be more difficult). As described in the methods, the data required extensive cleanup and formatting to be able to drill down to specific flags for patient specimens.

Operational improvements were facilitated by our analysis. The two main changes that were implemented based on the autoverification analysis were to increase the IG flag cutoff requiring manual review from 2% to 5% and to switch to the PLT-F method for all PLT counts. Ironically, the default manufacturer recommendation for the IG flag of 5% was a choice that minimized unnecessary manual intervention, as we did not identify any clear clinical advantage in the lower threshold that had been set based on experience with an earlier generation of hematology analyzer. The autoverification analysis related to platelets demonstrated the improved efficiency and lower rerun rates with the PLT-F method that can better distinguish between platelets and fragmented RBCs.<ref name=":10" /><ref name=":20" /><ref>{{Cite journal |last=Schapkaitz |first=E. |last2=Raburabu |first2=S. |date=2018-03 |title=Performance evaluation of the new measurement channels on the automated Sysmex XN-9000 hematology analyzer |url=https://pubmed.ncbi.nlm.nih.gov/29374555 |journal=Clinical Biochemistry |volume=53 |pages=132–138 |doi=10.1016/j.clinbiochem.2018.01.014 |issn=1873-2933 |pmid=29374555}}</ref><ref>{{Cite journal |last=Tantanate |first=Chaicharoen |last2=Khowawisetsut |first2=Ladawan |last3=Pattanapanyasat |first3=Kovit |date=2017-06 |title=Performance Evaluation of Automated Impedance and Optical Fluorescence Platelet Counts Compared With International Reference Method in Patients With Thalassemia |url=https://pubmed.ncbi.nlm.nih.gov/28402168 |journal=Archives of Pathology & Laboratory Medicine |volume=141 |issue=6 |pages=830–836 |doi=10.5858/arpa.2016-0222-OA |issn=1543-2165 |pmid=28402168}}</ref><ref>{{Cite journal |last=Tantanate |first=Chaicharoen |last2=Khowawisetsut |first2=Ladawan |last3=Sukapirom |first3=Kasama |last4=Pattanapanyasat |first4=Kovit |date=2019-05 |title=Analytical performance of automated platelet counts and impact on platelet transfusion guidance in patients with acute leukemia |url=https://pubmed.ncbi.nlm.nih.gov/30761915 |journal=Scandinavian Journal of Clinical and Laboratory Investigation |volume=79 |issue=3 |pages=160–166 |doi=10.1080/00365513.2019.1576100 |issn=1502-7686 |pmid=30761915}}</ref> Given that our laboratory receives many pediatric samples, including from hematology/oncology patients, use of PLT-F minimized repeat analysis for specimens that often contain low sample volumes. The rule changes reported in the present study have now been in place, and we are not aware of any clinical issues arising from these changes.

Future directions to pursue include the development of software that more easily enables analysis of autoverification rates and the impact of specific rules and flags. This may be with commercial vendor and/or home-grown software development. A data warehouse is a possibility. In the present study, such a warehouse would need to be able to access the DI database, or the DI database would need to be regularly duplicated to a different server. To allow for reliable evaluation of auto-verification, the data warehouse would ideally have discrete data for specimen comments/flags and operator identification (which could indicate manual versus auto-verification). One practical challenge to this approach would be trying to avoid causing latency issues on the production server. Given limited resources and competing informatics projects, we have not yet pursued such a project. For laboratories seeking to further increase autoverification rates, even identifying one or two rules associated with a high rate of triggering manual review may allow for a significant increase in autoverification while maintaining high quality patient care.

==Acknowledgements==
The authors would like to thank staff within the University of Iowa Hospitals and Clinics Department of Pathology core laboratory and the University of Iowa Health Care Information Systems who helped provide the support for autoverification, middleware, and laboratory information system issues over the years.

===Financial support===
None.

===Conflict of interest===
There are no conflicts of interest.

==References==
{{Reflist|colwidth=30em}}

==Notes==
This presentation is faithful to the original, with only a few minor changes to presentation, spelling, and grammar. In some cases important information was missing from the references, and that information was added. The original document mentions an "Additional File 1"; however, the original doesn't appear to include that file. The reference to Additional File 1 is maintained for this version, but contact the author to acquire the file.

<!--Place all category tags here-->
[[Category:LIMSwiki journal articles (added in 2021)]]
[[Category:LIMSwiki journal articles (all)]]
[[Category:LIMSwiki journal articles on laboratory informatics]]
[[Category:LIMSwiki journal articles on clinical informatics]]
[[Category:LIMSwiki journal articles on software]]