Difference between revisions of "Journal:Advancing laboratory medicine in hospitals through health information exchange: A survey of specialist physicians in Canada"

From LIMSWiki
Jump to navigationJump to search
(Saving and adding more.)
(Saving and adding more.)
Line 372: Line 372:
{|  
{|  
  | STYLE="vertical-align:top;"|
  | STYLE="vertical-align:top;"|
{| class="wikitable" border="1" cellpadding="5" cellspacing="0" width="100%"
{| class="wikitable" border="1" cellpadding="5" cellspacing="0" width="80%"
  |-
  |-
   | style="background-color:white; padding-left:10px; padding-right:10px;" colspan="2"|'''Table 4.''' Perceived benefits from specialist physicians’ use of HIE for laboratory medicine; <sup>a</sup> As perceived by the specialist physicians on Likert scales of 1 (strongly disagree) to 5 (strongly agree), <sup>b</sup> Quebec Health Record
   | style="background-color:white; padding-left:10px; padding-right:10px;" colspan="2"|'''Table 4.''' Perceived benefits from specialist physicians’ use of HIE for laboratory medicine; <sup>a</sup> As perceived by the specialist physicians on Likert scales of 1 (strongly disagree) to 5 (strongly agree), <sup>b</sup> Quebec Health Record
Line 438: Line 438:
|}
|}
|}
|}
Component-based structural equation modeling (SEM) was used to explore empirically the causal paths implied by our research framework. The partial least squares (PLS) method was thus selected because it is better suited to measurement models such as ours that include both exogenous and endogenous “formative” constructs<ref name="RobertsConcept09">{{cite journal |title=Conceptualizing and testing formative constructs: tutorial and annotated example |journal=ACM SIGMIS Database |author=Roberts, N.; Thatcher, J. |volume=40 |issue=3 |pages=9–39 |year=2009 |doi=10.1145/1592401.1592405}}</ref>, as presented in Fig. 2. As implemented in the SmartPLS software, this technique was also chosen for its robustness in terms of the distribution of residuals and its greater affinity for exploratory rather than confirmatory research purposes when compared to covariance-based SEM methods.<ref name="GefenAnUpd11">{{cite journal |titl=Editor's Comments: An Update and Extension to SEM Guidelines for Administrative and Social Science Research |journal=MIS Quarterly |author=Gefen, D.; Rigdon, E.E.; Straub, D. |volume=35 |issue=2 |pages=iii–xiv |year=2011 |doi=10.2307/23044042}}</ref>
[[File:Fig2 Raymond BMCMedInfoDecMak2020 20.png|800px]]
{{clear}}
{|
| STYLE="vertical-align:top;"|
{| border="0" cellpadding="5" cellspacing="0" width="800px"
|-
  | style="background-color:white; padding-left:10px; padding-right:10px;"| <blockquote>'''Figure 2.''' Path analysis of the use of HIE for laboratory medicine in hospitals (PLS, ''n'' = 566)</blockquote>
|-
|}
|}





Revision as of 18:58, 4 August 2020

Full article title Advancing laboratory medicine in hospitals through health information exchange: A survey of specialist physicians in Canada
Journal BMC Medical Informatics and Decision Making
Author(s) Raymond, Louis; Maillet, Éric; Trudel, Marie-Claude; Marsan, Josianne; de Guniea, Ana Ortiz; Paré, Guy
Author affiliation(s) Université du Québec à Trois-Rivières, Université de Sherbrooke, HEC Montréal, Université Laval
Primary contact Online contact form
Year published 2020
Volume and issue 20
Article # 44
DOI 10.1186/s12911-020-1061-z
ISSN 1472-6947
Distribution license Creative Commons Attribution 4.0 International
Website https://link.springer.com/article/10.1186/s12911-020-1061-z
Download https://link.springer.com/content/pdf/10.1186/s12911-020-1061-z.pdf (PDF)

Abstract

Background: Laboratory testing occupies a prominent place in healthcare. Information technology systems have the potential to empower laboratory experts and to enhance the interpretation of test results in order to better support physicians in their quest for better and safer patient care. This study sought to develop a better understanding of which laboratory information exchange (LIE) systems and features specialist physicians are using in hospital settings to consult their patients’ laboratory test results, and what benefit they derive from such use.

Methods: As part of a broader research program on the use of health information exchange systems for laboratory medicine in Quebec, Canada, this study was designed as on online survey. Our sample is composed of 566 specialist physicians working in hospital settings, out of the 1,512 physicians who responded to the survey (response rate of 17%). Respondents are representative of the targeted population of specialist physicians in terms of gender, age, and hospital location.

Results: We first observed that 80% of the surveyed physicians used the province-wide interoperable electronic health records (iEHR) system, and 93% used a laboratory results viewer (LRV) to consult laboratory test results, while most (72%) use both systems to retrieve lab results. Next, our findings reveal important differences in the capabilities available in each type of system and in the use of these capabilities. Third, there are differences in the nature of the perceived benefits obtained from the use of each of these two systems. Last, the extent of use of an LRV is strongly influenced by the IT artfact itself (i.e., the hospital’s LRV available capabilities), while the use of the provincial iEHR system is influenced by its organizational context (i.e., the hospital’s size and location).

Conclusions: The main contribution of this study lies in its insights into the role played by context in shaping physicians’ choices about which LIE systems to adopt, which features to use, and the different perceptions they have about benefits arising from such use. One related implication for practice is that success of LIE initiatives should not be solely assessed with basic usage statistics.

Keywords: laboratory information exchange, information systems, laboratory medicine, specialist physician, hospital, perceived benefits, online survey research

Background

Laboratory testing occupies a prominent place in healthcare.[1] For instance, more than seven billion laboratory tests are performed each year in the United States.[2] It is also reported that about 70% of all medical decisions are based on laboratory test results.[3] In hospital settings, which are the focus of the present study, 98% of admitted patients have one or more laboratory tests prescribed.[4] To provide services across a broad continuum and to perform increasingly complex tests, laboratories require sophisticated medical technologies and highly qualified staff.[1] Faced with this growing complexity, treating physicians must be able to rely on consistent clinical support provided by laboratory medicine specialists such as radiologists and pathologists.[5][6]

A recent study found that among seven countries, Canada ranked second in terms of physician self-reported errors in laboratory and diagnostic processes, as well as delays in reporting abnormal results.[7] One way to improve the quality and safety of patient care is to emphasize prevention and error management using well-designed information technology (IT) systems.[8][9] Indeed, the laboratory testing process involves the constant exchange of information among patients, physicians, nurses, and laboratory specialists which, nowadays, is supported by multiple IT systems and platforms.[10]

Missing laboratory results may have considerable consequences for patients and are due to several factors, including the way systems and practices are used to monitor test results, how critical results are managed, and how care is transitioned across settings.[10] To prevent medical errors[8], medical laboratories have deployed laboratory information systems (LIS) with user-friendly interfaces, tracking tools, and electronic alerts[5][11]; computerized physician order entry (CPOE) systems; and clinical decision support systems.[12] These systems empower laboratory specialists to enhance the interpretation of test results in order to better support physicians in their quest for better and safer patient care.[5] Although physicians may have access to an LIS, these systems are primarily designed to meet the needs of laboratory personnel. Therefore, other laboratory information exchange (LIE) systems are required to improve the reliability of the laboratory testing process[13] and, hence, need to be integrated with other clinical information systems (CISs) physicians use in hospitals such as electronic health records (EHRs).[14][15]

Prior research in the information systems (IS) field draws two main conclusions that are pertinent to this study. First, the mere adoption of a given IT system is not enough to achieve improvements in performance.[16] In fact, prior investigations of the relationship between IT system use (i.e., duration or frequency of use) and individual and organizational performance outcomes have yielded contradictory and inconclusive results.[17][18][19] Instead, it appears that performance improvements depend more on how a given IT system is used than on for how long.[20][21] More precisely, recent research shows that the extended use of a given IT system (i.e., conceptualized as the extent to which system features are utilized) is positively related to performance outcomes.[22] Research in the medical informatics field has recently corroborated the relationship between extended use of a system and performance outcomes such as quality of care, efficiency, operational performance, and economic performance.[23] Second, the IS literature has, for the most part, failed to conceptualize the IT artifact objectively.[20] That is, instead of capturing the features available in a system, researchers have focused on mental representations of the system (e.g., perceived ease of use, perceived usefulness).[24][25] Such mental representations are not of practical use, as they do not give any information about how the capabilities available in a system shape its extended use, nor do they provide concrete feedback to system designers about the criticality of certain features or the need for additional ones. As a result, IT systems, such as LIE, need to be better conceptualized in terms of their key functionalities or features.

Considering the above, the present study pursues two main objectives. First, it sought to develop a better understanding of which LIE systems and features specialist physicians (SPs) working in hospital settings are using to consult their patients’ laboratory test results, and what benefits they derive from such usage. More precisely, we attempt to provide answers to the following research questions: What is the nature of LIE usage in hospitals, and what types of information systems and features are being used by SPs for laboratory medicine purposes? How extensive is this use? What are the benefits obtained by SPs from extended LIE usage? Second, this study attempts to identify the contextual factors that lead to or influence the extended use of LIE systems by SPs. While medical informatics researchers have investigated the facilitators and barriers related to the adoption of EHRs in hospital settings[26][27][28][29], to our knowledge no prior study has focused on the antecedents to LIE system usage per se. The present study attempts to fill this gap. As explained below, inspired by prior research on EHRs we investigated the individual, organizational, and IT artifactual antecedents to LIE use.

Methods

As shown in Fig. 1, a conceptual framework was developed to describe and explain SPs’ use of health information exchange (HIE) systems for laboratory medicine in hospital settings, as well as the potential antecedents and performance outcomes of such use. This framework guided the design of the survey administered to find answers to our research questions. The framework is founded on prior research on HIE use in hospital settings and on the impacts of such use on laboratory testing in particular.[5][15][29][30] Moreover, we followed Burton-Jones and Grange[16] in assuming that using HIE systems per se does not necessarily enable laboratory medicine in hospital care. Our conceptual framework thus implies that only an “extended” use of LIE systems can have a positive impact on the practice of laboratory medicine by SPs, in terms of their efficiency and the quality of the care services provided to their patients.[23]


Fig1 Raymond BMCMedInfoDecMak2020 20.png

Figure 1. Conceptual framework

As part of a broader research program on the use of HIE systems for laboratory medicine in the province of Quebec, Canada, this study was designed as an online survey. As described below, we followed best practices concerning web-based survey methodology.[31] The survey questionnaire was built following the previously mentioned review of the extant literature and a series of interviews with 25 physicians located in 11 different regions of Quebec. Survey respondents were recruited with the help of the Quebec’s Ministry of Health and Social Services, which emailed an invitation letter to the 9005 physicians who had authorized access to the province-wide interoperable electronic health record (iEHR), called the Quebec Health Record (QHR). The letter included a hyperlink and a QR code for mobile devices, directing respondents to access the survey questionnaire through a secure web page. Developed with the Qualtrics online survey platform[32], the survey instrument was first approved by the province’s health authorities and then pre-tested with 10 physicians. Each physician was interviewed about the questionnaire’s format and instructions, as well as the wording of questions and possible answers, to ensure that they were interpreted as intended by the researchers. Following a few minor adjustments to the survey instrument, the study received final approval from the ethics committee of each researcher’s institution. Two reminder letters were sent to all targeted physicians seven and 14 days after the initial invitation.

Our sample is composed of 566 SPs providing secondary or tertiary care in hospital settings, out of the 1,512 physicians who fully responded to the survey (for a 17% response rate). The potential for non-response bias was ascertained by comparing the 112 “late” respondents (i.e., those who answered after receiving the second reminder) with the 454 “early” respondents. No significant differences were found between these two groups, thus indicating the absence of such a bias. The data were then analyzed through descriptive statistics, Chi-squared analysis, structural equation modeling (using SmartPLS software), cluster analysis, and analysis of variance and covariance (using SPSS software). The internal validity of the two index measures of HIE use was ascertained with “item analysis,” in which we confirmed that each measure correlated sufficiently with its component items.[33] The internal validity of the two scale measures of the impacts of HIE systems use was tested with Cronbach’s α coefficient (> 0.6 threshold for exploratory research).

Results

As shown in Table 1 (see top section), 49% of the SPs in our sample were women. In terms of clinical experience, 34% had less than 10 years of experience, 35% had 10 to 24 years, and 31% had 25 years or more. All major medical specialties are represented, including psychiatry, anesthesiology, pediatrics, radiology, internal medicine, surgery, obstetrics-gynecology, cardiology, and others. Respondents were asked to indicate what their main work affiliation was and to describe their use of HIE systems in this context. All SPs practiced in hospital settings; 44% in small or medium-size establishments (1 to 149 specialists), and 56% in large ones (150 or more specialists). As to their location, 70% practiced in a hospital located in a central or urban region, whereas 30% worked in peripheral or rural regions. It is worth noting that our respondents are representative of the targeted population of SPs in terms of gender (46% are women), age (average is 49 years old) and location (65% work in hospitals located in central or urban regions).[a]

Table 1. Context of HIE use by specialist physicians (SPs) for laboratory medicine: a Quebec Health Record; b For viewing lab results provided by a LIS, a CIS, and/or a regional HIE platform; * The χ2 value indicates a significant difference (p < 0.05 | p < 0.001) between iEHR users and non-users
Characteristics of the SPs (user context) All specialist physicians iEHRa users iEHR non-users Chi-squared statistic
(N = 566) freq. (%) (n = 451) freq. (%) (n = 115) freq. (%)
Gender
   Female 278 (49.1) 221 (49.0) 57 (49.6) 0.0
   Male 288 (50.9) 230 (51.0) 58 (50.4)
Age
   29 years old or less 7 (1.2) 6 (1.3) 1 (0.9) 5.6
   30–49 years old 317 (56.0) 256 (56.8) 61 (53.1)
   50–59 years old 137 (24.2) 101 (22.4) 36 (31.3)
   60 years old or more 105 (18.6) 88 (19.5) 17 (14.8)
Clinical experience
   5 years or less 85 (15.0) 72 (16.0) 13 (11.3) 5.9
   5–9 years 107 (18.9) 85 (18.8) 22 (19.1)
   10–24 years 196 (34.6) 156 (34.6) 40 (34.8)
   25 years or more 178 (31.4) 138 (30.6) 40 (34.8)
Characteristics of the SPs’ hospital facility (organizational context) All specialist physicians iEHRa users iEHR non-users Chi-squared statistic
(N = 566) freq. (%) (n = 451) freq. (%) (n = 115) freq. (%)
Size of the hospital
   1-49 specialist physicians 110 (19.4) 84 (18.6) 26 (22.6) 1.1
   50–149 specialist physicians 139 (24.6) 111 (24.6) 28 (24.3)
   150–299 specialist physicians 170 (30.0) 136 (30.2) 34 (29.6)
   300–1500 specialist physicians 147 (26.0) 12 (26.6) 27 (23.5)
Location of the hospital
   Urban region 395 (69.8) 340 (75.4) 55 (47.8) 33.0*
   Rural region 171 (30.2) 111 (24.6) 60 (52.2)
HIE systems used by the SPs for laboratory medicine (IT artifactual context) All specialist physicians iEHRa users iEHR non-users Chi-squared statistic
(N = 566) freq. (%) (n = 451) freq. (%) (n = 115) freq. (%)
Laboratory Results Viewerb
   LRV non-user 42 (7.4) 42 (9.3) 0 (0.0) 11.6*
   LRV user 524 (92.6) 409 (90.7) 115 (100)

In terms of the health IT artefacts used for HIE purposes, Table 1 reveals that 451 (80%) SPs consult laboratory test results through the province-wide iEHR and 524 (93%) through a LRV. In simple terms, an LRV is a common interface that allows physicians to access test results from their hospital’s CIS, a public or private medical laboratory’s LIS, and/or their region’s HIE platform (RHIEP)[b].[34][35] Despite being labelled “viewer,” some LRVs also have CPOE features, thus allowing laboratory tests prescription. The iEHR[c] is deployed by the Quebec government within the context of Canada’s national healthcare system. [36]. It appears that significantly more of the SPs who do not use the province-wide iEHR practice medicine in rural regions.[36]

Table 2 presents the different types of HIE systems used by the surveyed physicians. In this regard, there appears to be three main HIE use cases: a first case in which a SP uses only the iEHR, a second case in which they use only an LRV, and a third case in which both types of HIE systems are used in combination. The third case is the most prevalent, as a large majority of the sampled SPs (72%) are found to retrieve lab results through both the iEHR and an LRV. However, it is noteworthy that the SPs may use an LRV but not the iEHR to order new lab tests. Conversely, only 28% of the SPs in our sample use a single source to retrieve lab test results, either the iEHR (8%) or an LRV (20%). Moreover, the SPs’ use of an LRV is quite varied in terms of the combination of systems (LIS, CIS, and RHIEP) that they access for lab purposes through the common interface provided by the hospital. For instance, 32% of LRV users access laboratory test results through both their hospital’s CIS and their regional HIE platform.


Table 2. Types of HIE systems used by specialist physicians for laboratory medicine; iEHR = interoperable electronic health record, LRV = laboratory results viewer, CIS = clinical information system, LIS = laboratory information system, RHIEP = regional HIE platform
HIE use case HIE system used Number of specialist physicians
I iEHR 42 (7.4%)
II LRV 115 (20.3%)
III iEHR & LRV 409 (72.3%)
Total 566 (100%)
LRV use case HIE system used Number of specialist physicians
i CIS 149 (28.4%)
ii LIS 55 (10.5%)
iii RHIEP 20 (3.8%)
iv CIS & LIS 66 (12.6%)
v CIS & RHIEP 170 (32.4%)
vi LIS & RHIEP 10 (1.9%)
vii CIS & LIS & RHIEP 54 (10.3%)
Total 524 (100%)

Table 3 reveals important differences in the HIE capabilities available in each type of system, LRV and iEHR, and in the actual use of these capabilities by SPs. For instance, the possibility of electronically requesting a laboratory analysis and printing identifying labels for the samples is a capability that is available in only 55% of the LRV systems consulted by SPs. Yet 48% of the SPs are using it, leaving only 7% of the SPs with access to the functionality not using it. However, SPs seem to use most of the HIE capabilities available to them, utilizing on average 81 and 77% of the consultation capabilities available in the iEHR and their LRV, respectively. A notable exception is that only 39% of LRV users access patients’ test results produced by the laboratories in their region, even though this capability is available in 89% of the LRV systems.

Table 3. HIE consultation capabilities for laboratory medicine used by specialist physicians; a Quebec Health Record
HIE capability for laboratory medicine Availability of HIE capability (% of systems) Use of HIE capability (% of SPs)
LRV consultation capabilities for laboratory medicine (n = 524)
The laboratory results viewer allows me to …
• access all laboratory test results of a patient, whether I prescribed such tests or not. 92.9% 91.8%
• only access those patients’ test results that are produced by my hospital’s laboratory. 95.0% 87.6%
• generate tables and graphs for the display and analysis of lab test results. 82.5% 79.4%
• apply search criteria in order to find the lab test results that meet my needs. 79.6% 78.7%
• access patients’ test results that are produced by the laboratories in my region. 88.5% 38.8%
• electronically request a laboratory analysis and print identifying labels for the samples. 55.3% 48.3%
iEHR consultation capabilities for laboratory medicine (n = 451)
I consult the laboratory results provided by the QHRa when …
• the patient has been seen by a physician in another health establishment in Quebec. 100% 98.9%
• the patient is unable to reliably report to me his or her recent laboratory test results or his or her present state of health. 100% 98.4%
• the patient’s laboratory test results that I require are not found in my usual information sources (CIS, paper reports, etc.). 100% 98.2%
• the patient has no medical record in the hospital. 100% 95.8%
• doing an outpatient clinical consultation. 100% 93.8%
• I do not know the patient. 100% 80.0%
• I am consulted for a patient under observation in the emergency room. 100% 78.3%
• I am consulted for an admitted patient at the hospitalization unit. 100% 76.0%
• I am consulted in relation to a surgical intervention, a treatment or a diagnostic exam (e.g., in the operating room, in oncology). 100% 63.9%
• a physician beckons my expertise and I must take note of the clinical case remotely. 100% 60.3%

The next set of results pertains to the performance outcomes of HIE use in hospitals for laboratory testing, i.e., to the benefits perceived by SPs in terms of their individual efficiency and the quality of the care provided to their patients. As indicated in Table 4, there are important differences in the nature of the benefits obtained from each of the two types of systems used by SPs and in the extent to which these benefits were realized. For LRV users, the most important benefits were the greater, quicker, and easier access to lab test results. For users of the province-wide iEHR platform, the most critical benefits for their practice include significant improvements in continuity of care and in their ability to make better clinical decisions.

Table 4. Perceived benefits from specialist physicians’ use of HIE for laboratory medicine; a As perceived by the specialist physicians on Likert scales of 1 (strongly disagree) to 5 (strongly agree), b Quebec Health Record
Benefits of HIE use for laboratory medicinea Mean (s.d.)
Benefits of LRV consultation (n = 524)
• It is quicker for me to access the viewer to consult patients’ previous lab test results than wait to receive their paper medical record. 4.2 (1.1)
• My patients’ lab test results are easier to consult in the viewer than in the paper medical record. 4.1 (0.9)
• The viewer provides most of the lab test results that I need to care for patients referred to me. 4.0 (1.1)
• As compared with the paper medical record, the follow-up of admitted patients’ lab test results when changing the patient surveillance team is made easier with the viewer. 3.8 (1.2)
• As most of my patients reside in the region, I have little use for the QHRb because the viewer provides me with most of the lab test results that I need. 2.7 (1.4)
• The viewer is very useful in allowing me to access test results produced by the public laboratories in my region. 2.6 (1.6)
Benefits of iEHR consultation (n = 451)
Accessing the laboratory test results provided by the QHR …
• improves the continuity of my patients’ care. 4.2 (0.8)
• provides me with results that I cannot obtain from my usual information sources (e.g., lab results viewer). 4.1 (0.8)
• allows me to make better clinical decisions. 4.0 (0.8)
• improves the way in which I evaluate patients. 3.8 (0.9)
• has reduced the duplication of the lab tests prescribed to my patients. 3.8 (1.0)
• prevents me from missing an important result. 3.7 (1.0)
• increases the safety of my patients’ care. 3.7 (0.9)
• allows me to intervene more rapidly and effectively with my patients. 3.6 (0.9)
• provides me with an overall view of my patients’ lab results (patients’ test history). 3.6 (1.0)
• allows me to ask for more advanced lab analyses whose results may be useful to other clinicians. 3.6 (1.0)
• provides support to my clinical research or my performance measurement activities. 1.6 (1.0)

Component-based structural equation modeling (SEM) was used to explore empirically the causal paths implied by our research framework. The partial least squares (PLS) method was thus selected because it is better suited to measurement models such as ours that include both exogenous and endogenous “formative” constructs[37], as presented in Fig. 2. As implemented in the SmartPLS software, this technique was also chosen for its robustness in terms of the distribution of residuals and its greater affinity for exploratory rather than confirmatory research purposes when compared to covariance-based SEM methods.[38]


Fig2 Raymond BMCMedInfoDecMak2020 20.png

Figure 2. Path analysis of the use of HIE for laboratory medicine in hospitals (PLS, n = 566)


Footnotes

  1. Source: https://www.fmsq.org/en/profession/repartition-des-effectifs-medicaux
  2. A RHIEP is a multi-sided platform. On the one side, hospitals which usually include laboratories along with major CIS systems, join the platform, and upload their patients’ data to the RHIEP’s database. On the other side, physicians query RHIEP’s database and download the available laboratory information.
  3. The iEHR system deployed in Quebec, called the Quebec Health Record or QHR, is a secure provincial tool that is used to collect, store, and release information about patients’ health. It is organized into three clinical domains: medications, laboratories, and medical imaging. The health information contained in the QHR can be released on request to authorized providers and bodies in accordance with their access authorization. For more information: https://www.quebec.ca/en/health/your-health-information/quebec-health-record/

References

  1. 1.0 1.1 Hilbert, T.; Kurec, A.; Lifshitz, M.S. (2017). "General Concepts and Administrative Issues". In McPherson, R.A.; Pincus, M.R.. Henry's Clinical Diagnosis and Management by Laboratory Methods (23rd ed.). Elsevier Health Sciences. pp. 2–10. ISBN 9780323295680. 
  2. American Clinical Laboratory Association. "Value of Lab Testing". https://www.acla.com/value-of-lab-testing/. Retrieved 16 May 2019. 
  3. Silverstein, M.D. (8 April 2003). "An Approach to Medical Errors and Patient Safety in Laboratory Services" (PDF). http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.597.741&rep=rep1&type=pdf. 
  4. Ngo, A.; Gandhi, P.; Miller, W.G. (2017). "Frequency that Laboratory Tests Influence Medical Decisions". Journal of Applied Laboratory Medicine 1 (4): 410–14. doi:10.1373/jalm.2016.021634. 
  5. 5.0 5.1 5.2 5.3 Walz, S.E.; Darcy, T.P. (2013). "Patient safety & post-analytical error". Clinics in Laboratory Medicine 33 (1): 183–94. doi:10.1016/j.cll.2012.10.001. PMID 23331737. 
  6. Laposata, M. (2019). Laboratory medicine: The diagnosis of disease in the clinical laboratory (3rd ed.). McGraw-Hill Medical. ISBN 9781260116793. 
  7. O'Hagan, J.; MacKinnon, N.J.; Persaud, D. et al. (2009). "Self-reported medical errors in seven countries: Implications for Canada". Healthcare Quarterly 12: 55–61. doi:10.12927/hcq.2009.20967. PMID 19667778. 
  8. 8.0 8.1 Feldman, S.S.; Buchalter, S.; Hayes, L.W. et al. (2018). "Health Information Technology in Healthcare Quality and Patient Safety: Literature Review". JMIR Medical Informatics 6 (2): e10264. doi:10.2196/10264. PMC PMC6006013. PMID 29866642. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6006013. 
  9. Lifshitz, M.S. (2017). "Optimizing Laboratory Workflow and Performance". In McPherson, R.A.; Pincus, M.R.. Henry's Clinical Diagnosis and Management by Laboratory Methods (23rd ed.). Elsevier Health Sciences. pp. 11–19. ISBN 9780323295680. 
  10. 10.0 10.1 Callen, J.; Georgiou, A.; Li, J. et al. (2011). "The safety implications of missed test results for hospitalised patients: A systematic review". BMJ Quality and Safety 20 (2): 194–9. doi:10.1136/bmjqs.2010.044339. PMC PMC3038104. PMID 21300992. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3038104. 
  11. Etchells, E.; Adkhikari, N.K.J.; Wu, R. et al. (2011). "Real-time automated paging and decision support for critical laboratory abnormalities". BMJ Quality and Safety 20 (11): 924–30. doi:10.1136/bmjqs.2010.051110. PMID 21725046. 
  12. Procop, G.W.; Yerian, L.M.; Wyllie, R. et al. (2014). "Duplicate laboratory test reduction using a clinical decision support tool". American Journal of Clinical Pathology 141 (5): 718–23. doi:10.1309/AJCPOWHOIZBZ3FRW. PMID 24713745. 
  13. Murphy, D.R.; Satterly, T.; Rogith, D.. et al. (2019). "Barriers and facilitators impacting reliability of the electronic health record-facilitated total testing process". International Journal of Medical Informatics 127: 102–08. doi:10.1016/j.ijmedinf.2019.04.004. PMID 31128821. 
  14. Kaelber, D.C.; Bates, D.W. (2007). "Health information exchange and patient safety". Journal of Biomedical Informatics 40 (6 Suppl.): S40-5. doi:10.1016/j.jbi.2007.08.011. PMID 17950041. 
  15. 15.0 15.1 Wilkerson, M.L.; Henricks, W.H.; Casellani, W.J. et al. (2015). "Management of laboratory data and information exchange in the electronic health record". Archives of Pathology and Laboratory Medicine 139 (3): 319–27. doi:10.5858/arpa.2013-0712-SO. PMID 25724028. 
  16. 16.0 16.1 Burton-Jones, A.; Grange, C. (2013). "From Use to Effective Use: A Representation Theory Perspective". Information Systems Research 24 (3): 499–882. doi:10.1287/isre.1120.0444. 
  17. Deravaj, S.; Kohli, R. (2003). "Performance Impacts of Information Technology: Is Actual Usage the Missing Link?". Management Science 49 (3): 255–350. doi:10.1287/mnsc.49.3.273.12736. 
  18. Lucas Jr., H.C.; Spitler, V.K. (1999). "Technology Use and Performance: A Field Study of Broker Workstations". Decision Sciences 30 (2): 291–311. doi:10.1111/j.1540-5915.1999.tb01611.x. 
  19. "Use and Productivity in Personal Computing: An Emperical Test". ICIS 1989 Proceedings: 1. 1989. https://aisel.aisnet.org/icis1989/1. 
  20. 20.0 20.1 de Guinea, A.O.; Webster, J. (2013). "An Investigation of Information Systems Use Patterns: Technological Events as Triggers, the Effect of Time, and Consequences for Performance". MIS Quarterly 37 (4): 1165–88. doi:10.25300/MISQ/2013/37.4.08. 
  21. Maillet, É.; Mathieu, L.; Sicotte, C. (2015). "Modeling factors explaining the acceptance, actual use and satisfaction of nurses using an Electronic Patient Record in acute care settings: An extension of the UTAUT". International Journal of Medical Informatics 84 (1): 36–47. doi:10.1016/j.ijmedinf.2014.09.004. PMID 25288192. 
  22. Burton-Jones, A.; Straub Jr., D.W. (2006). "Reconceptualizing System Usage: An Approach and Empirical Test". Information Systems Research 17 (3): 197–325. doi:10.1287/isre.1060.0096. 
  23. 23.0 23.1 Raymond, L.; Paré, G.; de Guinea, A.O. et al. (2015). "Improving performance in medical practices through the extended use of electronic medical record systems: A survey of Canadian family physicians". BMC Medical Informatics and Decision Making 15: 27. doi:10.1186/s12911-015-0152-8. PMC PMC4397686. PMID 25888991. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4397686. 
  24. Abdullah, F.; Ward, R.; Ahmed, E. (2016). "Investigating the influence of the most commonly used external variables of TAM on students’ Perceived Ease of Use (PEOU) and Perceived Usefulness (PU) of e-portfolios". Computers in Human Behavior 63: 75–90. doi:10.1016/j.chb.2016.05.014. 
  25. Venkatesh, V. (2000). "Determinants of Perceived Ease of Use: Integrating Control, Intrinsic Motivation, and Emotion into the Technology Acceptance Model". Information Systems Research 11 (4): i–432. doi:10.1287/isre.11.4.342.11872. 
  26. Kruse, C.S.; Kothman, K.; Anerobi, K. et al. (2016). "Adoption Factors of the Electronic Health Record: A Systematic Review". JMIR Medical Informatics 4 (2): e19. doi:10.2196/medinform.5525. PMC PMC4909978. PMID 27251559. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4909978. 
  27. Adler-Milstein, J.; DesRoaches, C.M.; Kralovec, P. et al. (2015). "Electronic Health Record Adoption In US Hospitals: Progress Continues, But Challenges Persist". Health Affairs 34 (12): 2174-80. doi:10.1377/hlthaff.2015.0992. PMID 26561387. 
  28. Thompson, M.P.; Graetz, I. (2019). "Hospital adoption of interoperability functions". Healthcare 7 (3): 100347. doi:10.1016/j.hjdsi.2018.12.001. PMID 30595558. 
  29. 29.0 29.1 Walker, D.M. (2018). "Does participation in health information exchange improve hospital efficiency?". Health Care Management Science 21 (3): 426-438. doi:10.1007/s10729-017-9396-4. PMC PMC5568978. PMID 28236178. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5568978. 
  30. Holmgren, A.J.; Patel, V.; Adler-Milstein, J. (2017). "Progress In Interoperability: Measuring US Hospitals' Engagement In Sharing Patient Data". Health Affairs 36 (10): 1820—27. doi:10.1377/hlthaff.2017.0546. PMID 28971929. 
  31. Cunningham, C.T.; Quan, H. Hemmelgarn, B. et al. (2015). "Exploring physician specialist response rates to web-based surveys". BMC Medical Research Methodology 15: 32. doi:10.1186/s12874-015-0016-z. PMC PMC4404667. PMID 25888346. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4404667. 
  32. Snow, J. (2012). How to Use Qualtrics: Handbook for Research Professionals. Qualtrics Labs. ISBN 9780984932801. 
  33. Babbie, E.R. (2013). The Practice of Social Research. Cengage Learning. ISBN 9781133049791. 
  34. Bae, J.; Hockenberry, J.M.; Rask, K.J. et al. (2017). "Evidence that electronic health records can promote physician counseling for healthy behaviors". Health Care Management Review 42 (3): 258–68. doi:10.1097/HMR.0000000000000108. PMID 27050926. 
  35. Kreuter, J.M.; Basch, P. (2007). "A clinician-centric medical results viewer". Health Management Technology 28 (2): 56, 58, 60. PMID 17340965. 
  36. Giokas, D. (2005). "Canada Health Infoway - Towards a National Interoperable Electronic Health Record (EHR) Solution". Studies in Health Technology and Informatics 115: 108–40. PMID 16160221. 
  37. Roberts, N.; Thatcher, J. (2009). "Conceptualizing and testing formative constructs: tutorial and annotated example". ACM SIGMIS Database 40 (3): 9–39. doi:10.1145/1592401.1592405. 
  38. Gefen, D.; Rigdon, E.E.; Straub, D. (2011). MIS Quarterly 35 (2): iii–xiv. doi:10.2307/23044042. 

Notes

This presentation is faithful to the original, with only a few minor changes to presentation, grammar, and punctuation. In some cases important information was missing from the references, and that information was added. To more easily differentiate footnotes from references, the original footnotes (which where numbered) were updated to use lowercase letters.