Difference between revisions of "Journal:Laboratory information system – Where are we today?"

From LIMSWiki
Jump to navigationJump to search
(Saving and adding more.)
(Finished editing rest of content.)
 
(2 intermediate revisions by the same user not shown)
Line 19: Line 19:
|download    = [https://www.degruyter.com/downloadpdf/j/jomb.2017.36.issue-3/jomb-2017-0021/jomb-2017-0021.xml https://www.degruyter.com/downloadpdf/j/jomb.2017.36.issue-3/jomb-2017-0021/jomb-2017-0021.xml] (PDF)
|download    = [https://www.degruyter.com/downloadpdf/j/jomb.2017.36.issue-3/jomb-2017-0021/jomb-2017-0021.xml https://www.degruyter.com/downloadpdf/j/jomb.2017.36.issue-3/jomb-2017-0021/jomb-2017-0021.xml] (PDF)
}}
}}
{{ombox
 
| type      = content
Wider implementation of [[laboratory information system]]s (LIS) in [[Clinical laboratory|clinical laboratories]] in Serbia was initiated 10 years ago. The first LIS in the Railway Health Care Institute was implemented nine years ago. Before the LIS was initiated, manual admission procedures limited daily output of patients. Moreover, manual entering of patient data and ordering tests on analyzers was problematic and time-consuming. After completing tests, [[laboratory]] personnel had to write results in a patient register (with potential errors) and provide invoices for health insurance organizations. The first LIS brought forward some advantages with regards to these obstacles, but it also showed various weaknesses. These can be summarized as rigidity of the system and inability to fulfill user expectation. After four years of use, we replaced this system with another LIS. Hence, the main aim of this paper is to evaluate the advantages of using LIS in the Railway Health Care Institute's laboratory and also to discuss further possibilities for its application. After implementing an LIS, the admission process has proven to be much faster. The LIS enables electronic requests, barcoded specimens prevents identification errors, a bidirectional interface replaces redundant data entry steps, QC data are transferred automatically, results are electronically validated and automatically archived in the database, billing information is transferred electronically, and more. We also use some advanced options, like delta check, an HIL feature, quality indicators and various types of reports. All steps in total testing process are drastically improved after the implementation of an LIS, which had a positive impact on the quality of issued laboratory results. However, we expect development of new features in the future, for example auto-verification and inventory management. Using Railway Health Care Institute's laboratory as an example, we show that it is crucial that laboratory specialists have the main role in defining the desirable characteristics of an LIS which the institution aims to buy. This paper suggests that the main features of an LIS should be system flexibility and the capability of adjusting it to user needs and requests.
| style    = width: 500px;
| text      = This article should not be considered complete until this message box has been removed. This is a work in progress.
}}
Wider implementation of [[laboratory information system]]s (LIS) in [[Clinical laboratory|clinical laboratories]] in Serbia has been initiated ten years ago. The first LIS in the Railway Health Care Institute has been implemented nine years ago. Before the LIS was initiated, manual admission procedures limited daily output of patients. Moreover, manual entering of patients data and ordering tests on analyzers was problematic and time consuming. After completing tests, [[laboratory]] personnel had to write results in patient register (with potential errors) and provide invoices for health insurance organisation. First LIS brought forward some advantages with regards to these obstacles, but it also showed various weaknesses. These can be summarised in rigidity of system and inability to fulfil user expectation. After 4 years of use, we replaced this system with another LIS. Hence, the main aim of this paper is to evaluate advantages of using LIS in laboratory of the Railway Health Care Institute and also to discuss further possibilities for its application. After implementing LIS, admission procedure has proven to be much faster. LIS enabled electronic requests, barcoded specimens prevent identification errors, bidirectional interface replaces redundant data entry steps, QC data are transferred automatically, results are electronically validated and automatically archived in data base, billing information is transferred electronically, and more. We also use some advanced options, like delta check, HIL feature, quality indicators and various types of reports. All steps in total testing process are drastically improved after the implementation of LIS, which had a positive impact on the quality of issued laboratory results. However, we expect development of some new features in the future, for example auto-verification and inventory management. On the example of the laboratory of the Railway Health Care Institute, we show that it is crucial that laboratory specialists have the main role in defining desirable characteristics of LIS which institution aims to buy. This paper suggests that the main feature of LIS should be the flexibility of system and capability of adjustment to user needs and requests.


'''Keywords''': laboratory information system, flexibility, advanced options
'''Keywords''': laboratory information system, flexibility, advanced options


==Background==
==Background==
Laboratory information system is a software which receives, processes and stores [[information]] generated by the laboratory workflow. It automates the workflow of all information related to total testing process.<ref name="JacksonClin07">{{cite book |chapter=Chapter 15: Clinical Laboratory Informatics |title=Tietz Fundamentals of Clinical Chemistry |edition=6th |author=Jackson, B.R.; Harrison, J.H. |editor=Burtis, C.A.; Ashwood, E.R.; Bruns, D.E. |publisher=Saunders |pages=239–48 |year=2007 |isbn=9781437719406}}</ref> Wider implementation of LIS in Serbia has gained momentum in last ten years. It is expected that LIS facilitates communication between laboratory and clinicians and enables faster delivery of patient reports.<ref name="YusofTowards16">{{cite journal |title=Towards an evaluation framework for Laboratory Information Systems |journal=Journal of Infection and Public Health |author=Yusof, M.M.; Arifin, A. |volume=9 |issue=6 |pages=766–73 |year=2016 |doi=10.1016/j.jiph.2016.08.014}}</ref> The main aim of this paper is to evaluate advantages which the laboratory of the Railway Health Care Institute experienced with the implementation of LIS, as well as to point out at some advanced options and possible further use of its features.
A laboratory information system is software which receives, processes and stores [[information]] generated by the laboratory workflow. It automates the workflow of all information related to the total testing process.<ref name="JacksonClin07">{{cite book |chapter=Chapter 15: Clinical Laboratory Informatics |title=Tietz Fundamentals of Clinical Chemistry |edition=6th |author=Jackson, B.R.; Harrison, J.H. |editor=Burtis, C.A.; Ashwood, E.R.; Bruns, D.E. |publisher=Saunders |pages=239–48 |year=2007 |isbn=9781437719406}}</ref> Wider implementation of LIS in Serbia has gained momentum in the last 10 years. It is expected that the LIS facilitates communication between laboratory and clinicians and enables faster delivery of patient reports.<ref name="YusofTowards16">{{cite journal |title=Towards an evaluation framework for Laboratory Information Systems |journal=Journal of Infection and Public Health |author=Yusof, M.M.; Arifin, A. |volume=9 |issue=6 |pages=766–73 |year=2016 |doi=10.1016/j.jiph.2016.08.014}}</ref> The main aim of this paper is to evaluate advantages which the laboratory of the Railway Health Care Institute experienced with the implementation of an LIS, as well as to point out at some advanced options and possible further use of its features.


==Workflow before LIS==
==Workflow before LIS==
In last few decades medical laboratories have experienced dramatic transformation, due to automation and development of information technology. This transformation can be presented on the example of the laboratory department in the Railway Health Care Institute. In the early 2000s, the Institute purchased high capacity automated analyzers for [[hematology]], [[clinical chemistry]] and immunochemistry. This represented a huge breakthrough at the time. However, from today’s point of view, there were significant imperfections and slowdowns in the working process. Here we point out at some tasks which wasted most of our time. In spite of high quality laboratory equipment, there were still too many manual procedures. Registration of patients was slow due to admission desk procedures which required hand writing of all patient data, as well as writing receipts for paid analyses. Admission desk represented a ‘bottleneck’ which limited daily number of patients, in spite of huge throughput capabilities of analyzers. Following that, all ordered tests were locally entered on analyzers. Thus, after obtaining results, patient reports were printed from each analyzer on individual slips of paper where different papers were attached in order to produce a final report for patient. This form of report was visually unacceptable and rather difficult to read due to different fonts and formats. After that, all results were rewritten by hand in paper protocols for permanent storage. At the afternoon shift, staff again went through all data and made invoices for health insurance organization. Tubes were manually marked with IDs, as well as sample cups, and samples were aliquoted from tubes to cups. These actions represented a weak point for identification error, caused by personnel fatigue or lack of concentration. The results were then reviewed by clinical chemistry specialist in printed form, without access to previous results. Progressively, we have recognized an emerging need for implementation of laboratory information system in our everyday practice.
In the last few decades medical laboratories have experienced dramatic transformation due to the automation and development of information technology. This transformation can be presented by example of the laboratory department in the Railway Health Care Institute. In the early 2000s, the Institute purchased high-capacity automated analyzers for [[hematology]], [[clinical chemistry]] and immunochemistry. This represented a huge breakthrough at the time. However, from today’s point of view, there were significant imperfections and slowdowns in the working process. Here we point out how some tasks wasted most of our time. In spite of using high-quality laboratory equipment, there were still too many manual procedures. Registration of patients was slow due to admission desk procedures, which required writing by hand all patient data, as well as writing receipts for paid analyses. The admission desk represented a "bottleneck" which limited the daily number of patients in spite of the huge throughput capabilities of analyzers. Following that, all ordered tests were locally entered on analyzers. Thus, after obtaining results, patient reports were printed from each analyzer on individual slips of paper, resulting in different slips being attached in order to produce a final report for the patient. This form of report was visually unacceptable and rather difficult to read due to different fonts and formats. After that, all results were rewritten by hand in paper protocols for permanent storage. During the afternoon shift, staff again went through all data and made invoices for health insurance organizations. Tubes were manually marked with IDs, as well as sample cups, and samples were aliquoted from tubes to cups. These actions represented a weak point for identification error, caused by personnel fatigue or lack of concentration. The results were then reviewed by a clinical chemistry specialist in printed form, without access to previous results. Progressively, we recognized an emerging need for the implementation of a laboratory information system in our everyday practice.


==Implementation of LIS==
==Implementation of LIS==
First LIS was installed in our laboratory in 2008. It brought some advantages in terms of accelerations of admission procedures, but it did not fulfil our expectations. In the meantime, the laboratory purchased the last generation of analyzers, which caused additional inadequacy with LIS: it did not follow analyzers’ possibilities and did not allow them to achieve their maximal performance. Moreover, LIS did not have capability to adjust to user needs. Its rigidity did not allow for further improvements, and it began to impede the efficiency of the laboratory. That made us aware that we had to acquire new LIS.
The first LIS was installed in our laboratory in 2008. It brought some advantages in terms of acceleration of admission procedures, but it did not fulfill our expectations. In the meantime, the laboratory purchased the latest generation of analyzers, which caused additional inadequacy with the LIS: it did not tap into the analyzers’ possibilities and did not allow them to achieve their maximum performance. Moreover, the LIS did not have the capability of adjusting to user needs. Its rigidity did not allow for further improvements, and it began to impede the efficiency of the laboratory. That made us aware that we had to acquire a new LIS.


==Replacing existing LIS with new one==
==Replacing existing LIS with new one==
Although literature data implies that laboratories change their LIS only every 10 to 20 years because it is an enormous undertaking<ref name="JacksonClin07" />, the laboratory of the Railway Health Care Institute had to do that only 4 years after the first installation. We defined clear list of requirements for the new LIS in order to avoid previously experienced obstacles. Our requirements were specific about each desired feature and function for each step of total testing process, from preanalytical to postanalytical phase. The following was crucial when choosing the appropriate LIS: laboratory specialists had to be deeply involved in the selection process, and particularly in direct discussion of each and single feature of LIS with vendor, or, even better, with software developer. In summer 2012, simultaneously with integration of clinical chemistry and immunochemistry module, the installation of new LIS was completed. Here we present some of the benefits of using the new LIS.
Although research literature implies that laboratories only change their LIS every 10 to 20 years because due to being an enormous undertaking<ref name="JacksonClin07" />, the laboratory of the Railway Health Care Institute had to do that four years after the first installation. We defined a clear list of requirements for the new LIS in order to avoid previously experienced obstacles. Our requirements were specific about each desired feature and function for each step of the total testing process, from preanalytical to postanalytical phase. The following was crucial when choosing the appropriate LIS: laboratory specialists had to be deeply involved in the selection process, and particularly in direct discussion of each and every feature of the LIS with the vendor, or, even better, with the software developer. In the summer of 2012, simultaneously with the integration of the clinical chemistry and immunochemistry modules, the installation of a new LIS was completed. Here we present some of the benefits of using the new LIS.


Firstly, the interface with hospital information system (HIS) enables use of electronic test request and eliminates paper request form, which drastically accelerated admission desk procedures. Electronic request enabled the quality of communication between laboratory and clinicians. Moreover, LIS performs automated printing of receipts and bills for analyses which patients pay at admission desk. Printing of patient’s informed consent for venipuncture is also automated.<ref name="Lima-OliveiraLab15">{{cite journal |title=Laboratory Diagnostics and Quality of Blood Collection |journal=Journal of Medical Biochemistry |author=Lima-Oliveira, G.; Lippi, G.; Salvagno, G.L. et al. |volume=34 |issue=3 |pages=288-294 |year=2015 |doi=10.2478/jomb-2014-0043 |pmid=28356839 |pmc=PMC4922344}}</ref> By entering patient’s information and test orders in LIS, system generates electronic invoice for health insurance organization, without any further need for additional actions of admission personnel. Complete electronic register of patients is printed at the end of the day, so that personnel is not required to manually write paper register in order to be in accordance with legislative.
First, the ability to interface with a [[hospital information system]] (HIS) enables the use of electronic test requests and eliminates paper request forms, which drastically accelerates admission desk procedures. Electronic requesting improves the quality of communication between laboratory and clinicians. Moreover, the LIS performs automated printing of receipts and bills for analyses, which patients pay at the admission desk. Printing of patient’s informed consent for venipuncture is also automated.<ref name="Lima-OliveiraLab15">{{cite journal |title=Laboratory Diagnostics and Quality of Blood Collection |journal=Journal of Medical Biochemistry |author=Lima-Oliveira, G.; Lippi, G.; Salvagno, G.L. et al. |volume=34 |issue=3 |pages=288-294 |year=2015 |doi=10.2478/jomb-2014-0043 |pmid=28356839 |pmc=PMC4922344}}</ref> By entering a patient’s information and test orders into the LIS, the system can then generate an electronic invoice for health insurance organizations, without any further need for additional actions of admission personnel. Complete electronic registration of patients is printed at the end of the day, meaning that personnel are not required to manually write paper registers in order to be in accordance with regulatory legislation.


All samples are signed with barcode labels, so we use primary tubes for test analyses. Additionally we ended the time consuming and error prone sample aliquoting to sample cups. Operators on laboratory analyzers are now fully concentrated on analytical phase, because local test ordering on analyzers is replaced by automated order transfer from LIS. Moreover, LIS has bidirectional interface with all analyzers, so there is no need for entering orders into analyzers manually, nor retyping results from analyzer into LIS. After tests completion, results are transmitted to LIS and available to biochemist for review and verification. For few analyses which have to be manually performed (urine sediment, FOBT), manual test result entry is facilitated by configured drop-down lists with offered values for each parameter. This makes a result input much faster and also prevents typing errors which were very frequent in previous LIS, and a reason for recall of reports during review.
All samples are signed with barcode labels, so we use primary tubes for test analyses. Additionally, we ended the time-consuming and error-prone sample aliquoting to sample cups. Operators of laboratory analyzers are now fully concentrated on the analytical phase, because local test ordering on analyzers is replaced by automated order transfer from the LIS. Moreover, the LIS has a bidirectional interface with all analyzers, so there is no need for entering orders into analyzers manually, nor retyping results from the analyzer into the LIS. After test completion, results are transmitted to the LIS and made available to biochemists for review and verification. For the few analyses which have to be manually performed (urine sediment, FOBT), manual test result entry is facilitated by configured drop-down lists with offered values for each parameter. This makes result input much faster and also prevents typing errors, which were very frequent in the previous LIS, and a reason for recall of reports during review.


Verification screen offers plenty of data. On the top of the screen there are demographic data about patient and his/her diagnosis. Then there is current report with highlighted results which fall outside of the reference range. When cursor is positioned on any measured parameter, three previous results for that parameter are displayed at the bottom of the screen. With simple button press, we can see all results for that parameter from all previous reports for that patient, which can be displayed numerically but also graphically. User can define delta check rules and there will be a pop-up window warning if the rules are violated. Also, we have configured critical values for some tests. If critical value is obtained for some patient, there will be red color alarm in LIS, which is visible to all logged users regardless of the part of the program they are currently using. System enables entering time of communication of critical value to the doctor, monitors turnaround time (TAT) and stores reports about all communicated critical values. In printed form, critical value is also specially marked. During verification, clinical chemistry specialist can also see results of quality control for each parameter.<ref name="BragaRole15">{{cite journal |title=Role and Responsibilities of Laboratory Medicine Specialists in the Verification OF Metrological Traceability of in vitro Medical Diagnostics |journal=Journal of Medical Biochemistry |author=Braga, F.; Infusino, I; Panteghini, M. |volume=34 |issue=3 |pages=282-287 |year=2015 |doi=10.1515/jomb-2015-0004 |pmid=28356838 |pmc=PMC4922343}}</ref>
The verification screen offers plenty of data. On the top of the screen there are demographic data about patient and his/her diagnosis. Then there is a current report with highlighted results which fall outside of the reference range. When the cursor is positioned on any measured parameter, three previous results for that parameter are displayed at the bottom of the screen. With a simple button press, we can see all results for that parameter from all previous reports for that patient, which can be displayed not only numerically but also graphically. The user can define delta check rules, and there will be a pop-up window warning if the rules are violated. Also, we have configured critical values for some tests. If a critical value is obtained for some patient, there will be a red color alarm in the LIS, which is visible to all logged-in users regardless of the part of the program they are currently using. The system enables entering the time of communication of critical values to the doctor, monitors turnaround time (TAT) and stores reports about all communicated critical values. In printed form, any critical value is also specially marked. During verification, a clinical chemistry specialist can also see results of quality control for each parameter.<ref name="BragaRole15">{{cite journal |title=Role and Responsibilities of Laboratory Medicine Specialists in the Verification OF Metrological Traceability of in vitro Medical Diagnostics |journal=Journal of Medical Biochemistry |author=Braga, F.; Infusino, I; Panteghini, M. |volume=34 |issue=3 |pages=282-287 |year=2015 |doi=10.1515/jomb-2015-0004 |pmid=28356838 |pmc=PMC4922343}}</ref>


Just as in patient results, all values of quality control are automatically transmitted to LIS, as QC files. Transfer of control values does not need any additional operator action, for example ordering in LIS or manual input of obtained values. LIS calculates running mean and standard deviation, and generates Levey-Jennings graph which can be seen from verification screen. Also, on verification screen we can see all values of repeated measurements, and we can choose which of them will be reported. On this screen we can also make comments for doctor or patient. These comments may be configured as typical, which occur frequently, but they can be also typed in a free form text. Moreover, we can make some technical comments which are visible just for laboratory personnel. After the validation is completed, reports are ready for printing or e-mailing. Also, validated results are automatically sent to electronic health record and are immediately available for the doctor.
Just as in patient results, all quality control values are automatically transmitted to the LIS as QC files. Transfer of control values does not need any additional operator action, for example ordering directly from the LIS or manual input of obtained values. The LIS calculates running mean and standard deviation and generates Levey-Jennings graphs which can be seen from the verification screen. Also, we can see all values of repeated measurements on the verification screen, and we can choose which of them will be reported. On this screen we can also make comments for the doctor or patient. These comments may be configured as typical, which occur frequently, but they can also be typed in as free-form text. Moreover, we can make some technical comments which are visible just for laboratory personnel. After validation is complete, reports are ready for printing or e-mailing. Also, validated results are automatically sent to an [[electronic health record]] and are immediately available for the doctor.


When the patient is admitted to laboratory as an emergency case, LIS generates color-inversed barcode labels for samples, and they are clearly visible to all participants in laboratory testing process. Analyzers recognize such labeled samples as ‘stat’, and they are specially marked on verification screen to be treated as priority through the whole testing process, respecting turnaround time. For all patients, priority or regular ones, LIS makes precise evidence of time in performing each step in the testing process, so TAT for each patient is easily obtained from LIS. LIS also offers a wide spectrum of reports about the laboratory tests performed. Reporting data can be grouped by analyzers, time intervals, diagnosis, reference range, sex, doctor, and more. Additionally, it offers a variety of financial reports.
When the patient is admitted to the laboratory as an emergency case, the LIS generates color-inversed barcode labels for samples, and they are clearly visible to all participants in the laboratory testing process. Analyzers recognize such labeled samples as "stat," and they are specially marked on the verification screen to be treated as priority through the entire testing process, respecting turnaround time. For all patients, priority or regular ones, the LIS maintains precise evidence of the time each step in the testing process is performed, so TAT for each patient is easily obtained from the LIS. The LIS also offers a wide spectrum of reports about the laboratory tests performed. Reporting data can be grouped by analyzers, time intervals, diagnosis, reference range, sex, doctor, and more. Additionally, it offers a variety of financial reports.


Access to the LIS is permitted only to the authorized staff with personal password. System continuously creates a log of all actions in the system identified by user, and all employees are aware that reports of all actions (with user names and exact time of performing actions) are periodically reviewed. Finally, LIS keeps a log of communication details with analyzers (ordering time, measuring time, time of results transmission). All these logs enable maximal traceability of results through laboratory.<ref name="BragaRole15" />
Access to the LIS is permitted only to authorized staff with a personal password. The system continuously creates a log of all actions in the system, identified by user, and all employees are aware that reports of all actions (with user names and exact time of performing actions) are periodically reviewed. Finally, the LIS keeps a log of communication details with analyzers (ordering time, measuring time, time of results transmission). All these logs enable maximum traceability of results throughout the laboratory.<ref name="BragaRole15" />


==Advanced options==
==Advanced options==
LIS offers some advanced options, where some of them were developed or upgraded in the laboratory of the Railway Health Care Institute to respond to our requests and needs. For example, there are serum indices, verification rules, quality indicators, and inventory management.
The LIS offers some advanced options, where some of them were developed or upgraded in the laboratory of the Railway Health Care Institute to respond to our requests and needs. For example, it features serum indices, verification rules, quality indicators, and inventory management.
 
The quality of laboratory results is highly dependent on sample quality, and that is why reliable detection of the presences of interferents is one of the crucial preanalytical steps used to decrease the number of laboratory errors. Visual estimation of hemolysis, icterus and lipemia is unreliable and time consuming, so it should be replaced by automated measuring of serum indices.<ref name="SimundicComp09">{{cite journal |title=Comparison of visual vs. automated detection of lipemic, icteric and hemolyzed specimens: Can we rely on a human eye? |journal=Clinical Chemistry and Laboratory Medicine |author=Simundic, A.M.; Nikolac, N.; Ivankovic, V. et al. |volume=47 |issue=11 |pages=1361-5 |year=2009 |doi=10.1515/CCLM.2009.306 |pmid=19778291}}</ref> The Railway Health Care Institute's laboratory is one of the first in Serbia to implement systematic automated sample interference testing, which means that all serum tubes are being tested for hemolysis, icterus and lipemia (HIL test) on a clinical chemistry analyzer. Achieving this kind of systematic approach is not possible without the support of an LIS. Our LIS offers the possibility for a user to define which analyte requires testing for one or more of these interferents. In such a way, the LIS alone orders an HIL test from the analyzer without any action of admission staff, or the analyzer operator. Results of serum indices are automatically transmitted from the analyzer to LIS and are available to clinical chemistry specialists for evaluation and decision making. Furthermore, we have configured some rules related to serum indices, such that the LIS generates pop-up windows which prevent verification if some interferent is present in a concentration which can significantly affect some of the tests which are performed for that patient. This approach has enhanced systematic and objective detecting of the presence of interferents in patient samples, as well as adequate further management of such samples and results. These approaches provide high-quality reports and additional safety for our patients.<ref name="DolciHarmon14">{{cite journal |title=Harmonization of automated hemolysis index assessment and use: Is it possible? |journal=Clinica Chimica Acta |author=Dolci, A.; Panteghini, M. |volume=432 |pages=38–43 |year=2014 |doi=10.1016/j.cca.2013.10.012 |pmid=24513329}}</ref>
 
Aside from HIL indices, rules of verification can be configured in relation to some other laboratory situations; for example, an LIS alert can warn and prevent the release of LDL calculated by Friedwald formula, if triglycerides are higher than 4.5 mmol/L. Hence, the LIS enables an auto-verification feature that builds up on rules and algorithms for verification. We have made some trial steps, but this is huge, delicate and challenging work. We expect that auto-verification is the future of our laboratory, and it will enable biochemists to stay focused on those results which require professional attention and additional evaluation.
 
The LIS can also facilitate monitoring of quality indicators in medical laboratories. The identification and use of effective quality indicators in all phases of the total testing process is an essential requirement for laboratory accreditation, and for a valuable risk management strategy.<ref name="PlebaniHarmon14">{{cite journal |title=Harmonization of pre-analytical quality indicators |journal=Biochemia Medica |author=Plebani, M.; Sciacovelli, L.; Aita, A.; Chiozza, M.L. |volume=24 |issue=1 |pages=105–13 |year=2014 |doi=10.11613/BM.2014.012 |pmid=24627719 |pmc=PMC3936970}}</ref> Most laboratory specialists are aware that quality indicators are very important for quality of laboratory results. But, paradoxically, participation in available quality indicator programs is limited.<ref name="PlebaniTheQual16">{{cite journal |title=The quality indicator paradox |journal=Clinical Chemistry and Laboratory Medicine |author=Plebani, M. |volume=54 |issue=7 |pages=1119–22 |year=2016 |doi=10.1515/cclm-2015-1080 |pmid=26641971}}</ref> We are aware that this is a difficult and demanding task. Nevertheless, the LIS can help us with this, enabling recording of a large number of quality indicators, and for some of them this is an automated process. For example: hemolysis in serum greater than 0.5 g/L of free hemoglobin, exceeded turnaround time for emergency requests, and exceeded time for communication of critical results. Our laboratory participated in the IFCC project "Model of quality indicators," and this has been facilitated mostly thanks to our LIS. Reports about absolute and relative frequency in defined time intervals are easily available for all recorded indicators.
 
An electronic stock management system is a feature we are still developing in our laboratory. It is conceptualized as an inventory management module through the LIS. When completed, it would enable monitoring of stock status for each article in the laboratory and monitoring of the expiry dates. Furthermore, it would facilitate ordering of inventory to prevent shortage, but also prevent overstock of any item in the laboratory. Thus, we expect that this feature will optimize inventory management and ensure product availability.
 
==Staff satisfaction==
Research literature shows that during implementation of new technology procedures 20 percent of employees support change, 60 percent are unsure of change and 20 percent dislike change.<ref name="VillaAutomat10">{{cite journal |title=Automation, Lean, Six Sigma: Synergies for Improving Laboratory Efficiency |journal=Journal of Medical Biochemistry |author=Villa, D. |volume=29 |issue=4 |pages=339-48 |year=2010 |doi=10.2478/v10011-010-0038-3}}</ref> However, the situation was completely different with the implementation of a new LIS in our laboratory. One-hundred percent of employees demonstrated readiness for change, due to daily improvements. Manual tasks were either reduced or completely eliminated at all positions by the introduction of the new LIS, which increased the pace of all procedures. Thus, all employees experienced benefits in their daily practice.


The quality of laboratory results is highly dependent on sample quality, and that is why reliable detection of interferents presence is one of the crucial preanalytical steps in order to decrease the number of laboratory errors. Visual estimation of hemolysis, icterus and lipemia is unreliable and time consuming, so it should be replaced by automated measuring of serum indices.<ref name="SimundicComp09">{{cite journal |title=Comparison of visual vs. automated detection of lipemic, icteric and hemolyzed specimens: Can we rely on a human eye? |journal=Clinical Chemistry and Laboratory Medicine |author=Simundic, A.M.; Nikolac, N.; Ivankovic, V. et al. |volume=47 |issue=11 |pages=1361-5 |year=2009 |doi=10.1515/CCLM.2009.306 |pmid=19778291}}</ref> The laboratory of the Railway Health Care Institute is one of the first in Serbia which has implemented systematic automated sample interference testing, which means that all serum tubes are being tested for hemolysis, icterus and lipemia (HIL test) on clinical chemistry analyzer. Achieving this kind of systematic approach is not possible without the support from LIS. Our LIS offers a possibility where user defines which analyte requires testing for one or more of these interferents. In such way, LIS alone orders HIL test from analyzer without any action of admission staff, or analyzer operator. Results of serum indices are automatically transmitted from the analyzer to LIS, and are available to clinical chemistry specialist for evaluation and decision making. Furthermore, we have configured some rules related to serum indices, so LIS generates pop-up windows which prevent verification if some of interferents is present in concentration which can significantly affect some of tests which are performed for that patient. This approach has enhanced systematic and objective detecting of presence of interferents in patient sample, as well as adequate further management of such samples and results. These performances provide high quality of reports and additional safety for our patients.<ref name="DolciHarmon14">{{cite journal |title=Harmonization of automated hemolysis index assessment and use: Is it possible? |journal=Clinica Chimica Acta |author=Dolci, A.; Panteghini, M. |volume=432 |pages=38–43 |year=2014 |doi=10.1016/j.cca.2013.10.012 |pmid=24513329}}</ref>
==Conclusion: Benefits of LIS implementation==
The implementation of a quality laboratory information system points out some of the benefits for the laboratory: increased pace of patient admission, prevention of sample identification errors, prevention of test translation errors, permanent results storage in electronic form, prevention of billing errors, improved time savings and better staff organization. It should be emphasized that readily accesible data available to biochemists during verification improves the quality of reported data and patient safety. Cutting out the manual entry of patient data and hand writing of test results, as well as the repetitive job of aliquoting samples, has enabled better redistribution of working tasks between the employees. This implies a step forward towards optimization of the total testing process.<ref name="AykalPreTest16">{{cite journal |title=Pre-Test and Post-Test Applications to Shape the Education of Phlebotomists in A Quality Management Program: An Experience in a Training Hospital |journal=Journal of Medical Biochemistry |author=Aykal, G.; Keşapli, M.; Aydin, Ö. et al. |volume=35 |issue=3 |pages=347-53 |year=2016 |doi=10.1515/jomb-2016-0011 |pmid=28356887 |pmc=PMC5346814}}</ref>


Besides for HIL indices, rules of verification can be configured in relation to some other laboratory situations: for example there is alert in LIS which prevents release of LDL calculated by Friedwald formula, if triglycerides are higher of 4.5 mmol/L. Hence, LIS enables autoverification feature that builds up on rules and algorithms for verification. We have made some trial steps, but this is huge, delicate and challenging job. We expect that autoverification is the future of our laboratory, and it will enable biochemist to stay focused on those results which require professional attention and additional evaluation.
Finally, it is very important to highlight the key feature of a good LIS: its flexibility, i.e., its ability of adjust to user needs and integrate with other systems (analyzers, HIS). The laboratory is a dynamic structure and a rigid LIS can limit its development. Hence, quality and close cooperation between software developers and laboratory specialists is necessary. It is very important that biochemists define desirable LIS characteristics and suggest continuous changes which the laboratory needs. The biochemist as a user of the information system has to define needs and expectations from the LIS clearly and precisely. Future successful applications of an LIS highly depends on the care with which details are specified. Moreover, it is necessary to engage staff to cooperate during the test period for each LIS change, but also notice and report potential problems. Only an approach which implies patient and continuous collaborative work of biochemists and software developers can create an ideal LIS that meets particular laboratory needs.<ref name="TheodorssonQuality16">{{cite journal |title=Quality Assurance in Clinical Chemistry: A Touch of Statistics and A Lot of Common Sense |journal=Journal of Medical Biochemistry |author=Theodorsson, E. |volume=35 |issue=2 |pages=103–12 |year=2016 |doi=10.1515/jomb-2016-0012 |pmid=28356868 |pmc=PMC5346785}}</ref> Thus, it holds potential to provide a stimulating environment for further development and success.


LIS can also facilitate monitoring of quality indicators in medical laboratories. The identification and use of effective quality indicators in all phases of the total testing process is an essential requirement for laboratory accreditation, and for a valuable risk management strategy.<ref name="PlebaniHarmon14">{{cite journal |title=Harmonization of pre-analytical quality indicators |journal=Biochemia Medica |author=Plebani, M.; Sciacovelli, L.; Aita, A.; Chiozza, M.L. |volume=24 |issue=1 |pages=105–13 |year=2014 |doi=10.11613/BM.2014.012 |pmid=24627719 |pmc=PMC3936970}}</ref> Most laboratory specialists are aware that quality indicators are very important for quality of laboratory results. But, paradoxically, participation in available quality indicators programs is limited.<ref name="PlebaniTheQual16">{{cite journal |title=The quality indicator paradox |journal=Clinical Chemistry and Laboratory Medicine |author=Plebani, M. |volume=54 |issue=7 |pages=1119–22 |year=2016 |doi=10.1515/cclm-2015-1080 |pmid=26641971}}</ref> We are aware that this is a difficult and demanding task. Nevertheless, LIS can help us with this. LIS enables recording of large number of quality indicators, and for some of them this is an automated process. For example: hemolysis in serum greater than 0.5 g/L of free hemoglobin, exceeded turnaround time for emergency requests, exceeded time for communication of critical results. Our laboratory participated in IFCC project “Model of quality indicators”, and this has been facilitated mostly thanks to LIS. Reports about absolute and relative frequency in defined time interval are easily available for all recorded indicators.
==Conflict of interest statement==
The authors stated that they have no conflicts of interest regarding the publication of this article.


==References==
==References==
Line 65: Line 74:


==Notes==
==Notes==
This presentation is faithful to the original, with only a few minor changes to presentation, including the addition of PMCID and DOI when they were missing from the original reference. No other modifications were made in accordance with the "no derivatives" portion of the distribution license.
This presentation is faithful to the original, with only a few minor changes to presentation, including the addition of PMCID and DOI when they were missing from the original reference. Grammar and spelling were updated for readability and should not constitute "sufficient new creativity to be copyrightable"; no other modifications were made in accordance with the "no derivatives" portion of the distribution license.


<!--Place all category tags here-->
<!--Place all category tags here-->

Latest revision as of 18:54, 6 September 2017

Full article title Laboratory information system – Where are we today?
Journal Journal of Medical Biochemistry
Author(s) Lukić, Vera
Author affiliation(s) Railway Healthcare Institute (Belgrade, Serbia)
Primary contact Email: veralukic dot lab at gmail com
Year published 2017
Volume and issue 36 (3)
Page(s) 220–224
DOI 10.1515/jomb-2017-0021
ISSN 1452-8266
Distribution license Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported
Website https://www.degruyter.com/view/j/jomb.2017.36.issue-3/jomb-2017-0021/jomb-2017-0021.xml
Download https://www.degruyter.com/downloadpdf/j/jomb.2017.36.issue-3/jomb-2017-0021/jomb-2017-0021.xml (PDF)

Wider implementation of laboratory information systems (LIS) in clinical laboratories in Serbia was initiated 10 years ago. The first LIS in the Railway Health Care Institute was implemented nine years ago. Before the LIS was initiated, manual admission procedures limited daily output of patients. Moreover, manual entering of patient data and ordering tests on analyzers was problematic and time-consuming. After completing tests, laboratory personnel had to write results in a patient register (with potential errors) and provide invoices for health insurance organizations. The first LIS brought forward some advantages with regards to these obstacles, but it also showed various weaknesses. These can be summarized as rigidity of the system and inability to fulfill user expectation. After four years of use, we replaced this system with another LIS. Hence, the main aim of this paper is to evaluate the advantages of using LIS in the Railway Health Care Institute's laboratory and also to discuss further possibilities for its application. After implementing an LIS, the admission process has proven to be much faster. The LIS enables electronic requests, barcoded specimens prevents identification errors, a bidirectional interface replaces redundant data entry steps, QC data are transferred automatically, results are electronically validated and automatically archived in the database, billing information is transferred electronically, and more. We also use some advanced options, like delta check, an HIL feature, quality indicators and various types of reports. All steps in total testing process are drastically improved after the implementation of an LIS, which had a positive impact on the quality of issued laboratory results. However, we expect development of new features in the future, for example auto-verification and inventory management. Using Railway Health Care Institute's laboratory as an example, we show that it is crucial that laboratory specialists have the main role in defining the desirable characteristics of an LIS which the institution aims to buy. This paper suggests that the main features of an LIS should be system flexibility and the capability of adjusting it to user needs and requests.

Keywords: laboratory information system, flexibility, advanced options

Background

A laboratory information system is software which receives, processes and stores information generated by the laboratory workflow. It automates the workflow of all information related to the total testing process.[1] Wider implementation of LIS in Serbia has gained momentum in the last 10 years. It is expected that the LIS facilitates communication between laboratory and clinicians and enables faster delivery of patient reports.[2] The main aim of this paper is to evaluate advantages which the laboratory of the Railway Health Care Institute experienced with the implementation of an LIS, as well as to point out at some advanced options and possible further use of its features.

Workflow before LIS

In the last few decades medical laboratories have experienced dramatic transformation due to the automation and development of information technology. This transformation can be presented by example of the laboratory department in the Railway Health Care Institute. In the early 2000s, the Institute purchased high-capacity automated analyzers for hematology, clinical chemistry and immunochemistry. This represented a huge breakthrough at the time. However, from today’s point of view, there were significant imperfections and slowdowns in the working process. Here we point out how some tasks wasted most of our time. In spite of using high-quality laboratory equipment, there were still too many manual procedures. Registration of patients was slow due to admission desk procedures, which required writing by hand all patient data, as well as writing receipts for paid analyses. The admission desk represented a "bottleneck" which limited the daily number of patients in spite of the huge throughput capabilities of analyzers. Following that, all ordered tests were locally entered on analyzers. Thus, after obtaining results, patient reports were printed from each analyzer on individual slips of paper, resulting in different slips being attached in order to produce a final report for the patient. This form of report was visually unacceptable and rather difficult to read due to different fonts and formats. After that, all results were rewritten by hand in paper protocols for permanent storage. During the afternoon shift, staff again went through all data and made invoices for health insurance organizations. Tubes were manually marked with IDs, as well as sample cups, and samples were aliquoted from tubes to cups. These actions represented a weak point for identification error, caused by personnel fatigue or lack of concentration. The results were then reviewed by a clinical chemistry specialist in printed form, without access to previous results. Progressively, we recognized an emerging need for the implementation of a laboratory information system in our everyday practice.

Implementation of LIS

The first LIS was installed in our laboratory in 2008. It brought some advantages in terms of acceleration of admission procedures, but it did not fulfill our expectations. In the meantime, the laboratory purchased the latest generation of analyzers, which caused additional inadequacy with the LIS: it did not tap into the analyzers’ possibilities and did not allow them to achieve their maximum performance. Moreover, the LIS did not have the capability of adjusting to user needs. Its rigidity did not allow for further improvements, and it began to impede the efficiency of the laboratory. That made us aware that we had to acquire a new LIS.

Replacing existing LIS with new one

Although research literature implies that laboratories only change their LIS every 10 to 20 years because due to being an enormous undertaking[1], the laboratory of the Railway Health Care Institute had to do that four years after the first installation. We defined a clear list of requirements for the new LIS in order to avoid previously experienced obstacles. Our requirements were specific about each desired feature and function for each step of the total testing process, from preanalytical to postanalytical phase. The following was crucial when choosing the appropriate LIS: laboratory specialists had to be deeply involved in the selection process, and particularly in direct discussion of each and every feature of the LIS with the vendor, or, even better, with the software developer. In the summer of 2012, simultaneously with the integration of the clinical chemistry and immunochemistry modules, the installation of a new LIS was completed. Here we present some of the benefits of using the new LIS.

First, the ability to interface with a hospital information system (HIS) enables the use of electronic test requests and eliminates paper request forms, which drastically accelerates admission desk procedures. Electronic requesting improves the quality of communication between laboratory and clinicians. Moreover, the LIS performs automated printing of receipts and bills for analyses, which patients pay at the admission desk. Printing of patient’s informed consent for venipuncture is also automated.[3] By entering a patient’s information and test orders into the LIS, the system can then generate an electronic invoice for health insurance organizations, without any further need for additional actions of admission personnel. Complete electronic registration of patients is printed at the end of the day, meaning that personnel are not required to manually write paper registers in order to be in accordance with regulatory legislation.

All samples are signed with barcode labels, so we use primary tubes for test analyses. Additionally, we ended the time-consuming and error-prone sample aliquoting to sample cups. Operators of laboratory analyzers are now fully concentrated on the analytical phase, because local test ordering on analyzers is replaced by automated order transfer from the LIS. Moreover, the LIS has a bidirectional interface with all analyzers, so there is no need for entering orders into analyzers manually, nor retyping results from the analyzer into the LIS. After test completion, results are transmitted to the LIS and made available to biochemists for review and verification. For the few analyses which have to be manually performed (urine sediment, FOBT), manual test result entry is facilitated by configured drop-down lists with offered values for each parameter. This makes result input much faster and also prevents typing errors, which were very frequent in the previous LIS, and a reason for recall of reports during review.

The verification screen offers plenty of data. On the top of the screen there are demographic data about patient and his/her diagnosis. Then there is a current report with highlighted results which fall outside of the reference range. When the cursor is positioned on any measured parameter, three previous results for that parameter are displayed at the bottom of the screen. With a simple button press, we can see all results for that parameter from all previous reports for that patient, which can be displayed not only numerically but also graphically. The user can define delta check rules, and there will be a pop-up window warning if the rules are violated. Also, we have configured critical values for some tests. If a critical value is obtained for some patient, there will be a red color alarm in the LIS, which is visible to all logged-in users regardless of the part of the program they are currently using. The system enables entering the time of communication of critical values to the doctor, monitors turnaround time (TAT) and stores reports about all communicated critical values. In printed form, any critical value is also specially marked. During verification, a clinical chemistry specialist can also see results of quality control for each parameter.[4]

Just as in patient results, all quality control values are automatically transmitted to the LIS as QC files. Transfer of control values does not need any additional operator action, for example ordering directly from the LIS or manual input of obtained values. The LIS calculates running mean and standard deviation and generates Levey-Jennings graphs which can be seen from the verification screen. Also, we can see all values of repeated measurements on the verification screen, and we can choose which of them will be reported. On this screen we can also make comments for the doctor or patient. These comments may be configured as typical, which occur frequently, but they can also be typed in as free-form text. Moreover, we can make some technical comments which are visible just for laboratory personnel. After validation is complete, reports are ready for printing or e-mailing. Also, validated results are automatically sent to an electronic health record and are immediately available for the doctor.

When the patient is admitted to the laboratory as an emergency case, the LIS generates color-inversed barcode labels for samples, and they are clearly visible to all participants in the laboratory testing process. Analyzers recognize such labeled samples as "stat," and they are specially marked on the verification screen to be treated as priority through the entire testing process, respecting turnaround time. For all patients, priority or regular ones, the LIS maintains precise evidence of the time each step in the testing process is performed, so TAT for each patient is easily obtained from the LIS. The LIS also offers a wide spectrum of reports about the laboratory tests performed. Reporting data can be grouped by analyzers, time intervals, diagnosis, reference range, sex, doctor, and more. Additionally, it offers a variety of financial reports.

Access to the LIS is permitted only to authorized staff with a personal password. The system continuously creates a log of all actions in the system, identified by user, and all employees are aware that reports of all actions (with user names and exact time of performing actions) are periodically reviewed. Finally, the LIS keeps a log of communication details with analyzers (ordering time, measuring time, time of results transmission). All these logs enable maximum traceability of results throughout the laboratory.[4]

Advanced options

The LIS offers some advanced options, where some of them were developed or upgraded in the laboratory of the Railway Health Care Institute to respond to our requests and needs. For example, it features serum indices, verification rules, quality indicators, and inventory management.

The quality of laboratory results is highly dependent on sample quality, and that is why reliable detection of the presences of interferents is one of the crucial preanalytical steps used to decrease the number of laboratory errors. Visual estimation of hemolysis, icterus and lipemia is unreliable and time consuming, so it should be replaced by automated measuring of serum indices.[5] The Railway Health Care Institute's laboratory is one of the first in Serbia to implement systematic automated sample interference testing, which means that all serum tubes are being tested for hemolysis, icterus and lipemia (HIL test) on a clinical chemistry analyzer. Achieving this kind of systematic approach is not possible without the support of an LIS. Our LIS offers the possibility for a user to define which analyte requires testing for one or more of these interferents. In such a way, the LIS alone orders an HIL test from the analyzer without any action of admission staff, or the analyzer operator. Results of serum indices are automatically transmitted from the analyzer to LIS and are available to clinical chemistry specialists for evaluation and decision making. Furthermore, we have configured some rules related to serum indices, such that the LIS generates pop-up windows which prevent verification if some interferent is present in a concentration which can significantly affect some of the tests which are performed for that patient. This approach has enhanced systematic and objective detecting of the presence of interferents in patient samples, as well as adequate further management of such samples and results. These approaches provide high-quality reports and additional safety for our patients.[6]

Aside from HIL indices, rules of verification can be configured in relation to some other laboratory situations; for example, an LIS alert can warn and prevent the release of LDL calculated by Friedwald formula, if triglycerides are higher than 4.5 mmol/L. Hence, the LIS enables an auto-verification feature that builds up on rules and algorithms for verification. We have made some trial steps, but this is huge, delicate and challenging work. We expect that auto-verification is the future of our laboratory, and it will enable biochemists to stay focused on those results which require professional attention and additional evaluation.

The LIS can also facilitate monitoring of quality indicators in medical laboratories. The identification and use of effective quality indicators in all phases of the total testing process is an essential requirement for laboratory accreditation, and for a valuable risk management strategy.[7] Most laboratory specialists are aware that quality indicators are very important for quality of laboratory results. But, paradoxically, participation in available quality indicator programs is limited.[8] We are aware that this is a difficult and demanding task. Nevertheless, the LIS can help us with this, enabling recording of a large number of quality indicators, and for some of them this is an automated process. For example: hemolysis in serum greater than 0.5 g/L of free hemoglobin, exceeded turnaround time for emergency requests, and exceeded time for communication of critical results. Our laboratory participated in the IFCC project "Model of quality indicators," and this has been facilitated mostly thanks to our LIS. Reports about absolute and relative frequency in defined time intervals are easily available for all recorded indicators.

An electronic stock management system is a feature we are still developing in our laboratory. It is conceptualized as an inventory management module through the LIS. When completed, it would enable monitoring of stock status for each article in the laboratory and monitoring of the expiry dates. Furthermore, it would facilitate ordering of inventory to prevent shortage, but also prevent overstock of any item in the laboratory. Thus, we expect that this feature will optimize inventory management and ensure product availability.

Staff satisfaction

Research literature shows that during implementation of new technology procedures 20 percent of employees support change, 60 percent are unsure of change and 20 percent dislike change.[9] However, the situation was completely different with the implementation of a new LIS in our laboratory. One-hundred percent of employees demonstrated readiness for change, due to daily improvements. Manual tasks were either reduced or completely eliminated at all positions by the introduction of the new LIS, which increased the pace of all procedures. Thus, all employees experienced benefits in their daily practice.

Conclusion: Benefits of LIS implementation

The implementation of a quality laboratory information system points out some of the benefits for the laboratory: increased pace of patient admission, prevention of sample identification errors, prevention of test translation errors, permanent results storage in electronic form, prevention of billing errors, improved time savings and better staff organization. It should be emphasized that readily accesible data available to biochemists during verification improves the quality of reported data and patient safety. Cutting out the manual entry of patient data and hand writing of test results, as well as the repetitive job of aliquoting samples, has enabled better redistribution of working tasks between the employees. This implies a step forward towards optimization of the total testing process.[10]

Finally, it is very important to highlight the key feature of a good LIS: its flexibility, i.e., its ability of adjust to user needs and integrate with other systems (analyzers, HIS). The laboratory is a dynamic structure and a rigid LIS can limit its development. Hence, quality and close cooperation between software developers and laboratory specialists is necessary. It is very important that biochemists define desirable LIS characteristics and suggest continuous changes which the laboratory needs. The biochemist as a user of the information system has to define needs and expectations from the LIS clearly and precisely. Future successful applications of an LIS highly depends on the care with which details are specified. Moreover, it is necessary to engage staff to cooperate during the test period for each LIS change, but also notice and report potential problems. Only an approach which implies patient and continuous collaborative work of biochemists and software developers can create an ideal LIS that meets particular laboratory needs.[11] Thus, it holds potential to provide a stimulating environment for further development and success.

Conflict of interest statement

The authors stated that they have no conflicts of interest regarding the publication of this article.

References

  1. 1.0 1.1 Jackson, B.R.; Harrison, J.H. (2007). "Chapter 15: Clinical Laboratory Informatics". In Burtis, C.A.; Ashwood, E.R.; Bruns, D.E.. Tietz Fundamentals of Clinical Chemistry (6th ed.). Saunders. pp. 239–48. ISBN 9781437719406. 
  2. Yusof, M.M.; Arifin, A. (2016). "Towards an evaluation framework for Laboratory Information Systems". Journal of Infection and Public Health 9 (6): 766–73. doi:10.1016/j.jiph.2016.08.014. 
  3. Lima-Oliveira, G.; Lippi, G.; Salvagno, G.L. et al. (2015). "Laboratory Diagnostics and Quality of Blood Collection". Journal of Medical Biochemistry 34 (3): 288-294. doi:10.2478/jomb-2014-0043. PMC PMC4922344. PMID 28356839. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4922344. 
  4. 4.0 4.1 Braga, F.; Infusino, I; Panteghini, M. (2015). "Role and Responsibilities of Laboratory Medicine Specialists in the Verification OF Metrological Traceability of in vitro Medical Diagnostics". Journal of Medical Biochemistry 34 (3): 282-287. doi:10.1515/jomb-2015-0004. PMC PMC4922343. PMID 28356838. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4922343. 
  5. Simundic, A.M.; Nikolac, N.; Ivankovic, V. et al. (2009). "Comparison of visual vs. automated detection of lipemic, icteric and hemolyzed specimens: Can we rely on a human eye?". Clinical Chemistry and Laboratory Medicine 47 (11): 1361-5. doi:10.1515/CCLM.2009.306. PMID 19778291. 
  6. Dolci, A.; Panteghini, M. (2014). "Harmonization of automated hemolysis index assessment and use: Is it possible?". Clinica Chimica Acta 432: 38–43. doi:10.1016/j.cca.2013.10.012. PMID 24513329. 
  7. Plebani, M.; Sciacovelli, L.; Aita, A.; Chiozza, M.L. (2014). "Harmonization of pre-analytical quality indicators". Biochemia Medica 24 (1): 105–13. doi:10.11613/BM.2014.012. PMC PMC3936970. PMID 24627719. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3936970. 
  8. Plebani, M. (2016). "The quality indicator paradox". Clinical Chemistry and Laboratory Medicine 54 (7): 1119–22. doi:10.1515/cclm-2015-1080. PMID 26641971. 
  9. Villa, D. (2010). "Automation, Lean, Six Sigma: Synergies for Improving Laboratory Efficiency". Journal of Medical Biochemistry 29 (4): 339-48. doi:10.2478/v10011-010-0038-3. 
  10. Aykal, G.; Keşapli, M.; Aydin, Ö. et al. (2016). "Pre-Test and Post-Test Applications to Shape the Education of Phlebotomists in A Quality Management Program: An Experience in a Training Hospital". Journal of Medical Biochemistry 35 (3): 347-53. doi:10.1515/jomb-2016-0011. PMC PMC5346814. PMID 28356887. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5346814. 
  11. Theodorsson, E. (2016). "Quality Assurance in Clinical Chemistry: A Touch of Statistics and A Lot of Common Sense". Journal of Medical Biochemistry 35 (2): 103–12. doi:10.1515/jomb-2016-0012. PMC PMC5346785. PMID 28356868. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5346785. 

Notes

This presentation is faithful to the original, with only a few minor changes to presentation, including the addition of PMCID and DOI when they were missing from the original reference. Grammar and spelling were updated for readability and should not constitute "sufficient new creativity to be copyrightable"; no other modifications were made in accordance with the "no derivatives" portion of the distribution license.