Difference between revisions of "Journal:The effect of a test ordering software intervention on the prescription of unnecessary laboratory tests - A randomized controlled trial"

From LIMSWiki
Jump to navigationJump to search
(Saving and adding more.)
(Saving and adding more.)
Line 29: Line 29:
'''Methods''': Participants were family physicians working and prescribing diagnostic and [[laboratory]] tests.
'''Methods''': Participants were family physicians working and prescribing diagnostic and [[laboratory]] tests.


The intervention group had modified software with basic shortcut menu changes, where some tests were withdrawn or added, and with the implementation of an evidence-based [[Clinical decision support system|decision support system]] based on United States Preventive Services Task Force (USPSTF) recommendations. This intervention group was compared with typically used software (control group).
The intervention group had modified software with basic shortcut menu changes, where some tests were withdrawn or added, and with the implementation of an evidence-based [[clinical decision support system]] based on United States Preventive Services Task Force (USPSTF) recommendations. This intervention group was compared with typically used software (control group).


The outcomes were the number of tests prescribed from those: withdrawn from the basic menu; added to the basic menu; marked with green dots (USPSTF’s grade A and B); and marked with red dots (USPSTF’s grade D).
The outcomes were the number of tests prescribed from those: withdrawn from the basic menu; added to the basic menu; marked with green dots (USPSTF’s grade A and B); and marked with red dots (USPSTF’s grade D).
Line 49: Line 49:


To achieve better patient safety standards and improve healthcare system cost-effectiveness, there has been a worldwide effort to implement an integrated EHR system with diagnostic and [[Computerized physician order entry|laboratory test ordering communication systems]].<ref name="MoenEHealth13">{{cite journal |title=eHealth in Europe - Status and challenges |journal=Yearbook of Medical Informatics |author=Moen, A.; Hackl, W.O.; Hofdijk, J. et al. |volume=8 |pages=59–63 |year=2013 |pmid=23974549}}</ref><ref name="OlssonEuropean04">{{cite journal |title=European Commission activities in eHealth |journal=International Journal of Circumpolar Health |author=Olsson, S.; Lymberis, A.; Whitehouse, D. |volume=63 |issue=4 |pages=310-6 |year=2004 |pmid=15709306}}</ref><ref name="IakovidisEHealth08">{{cite journal |title=eHealth in Europe: From vision to reality |journal=Studies in Health Technology and Informatics |author=Iakovidis, I.; Purcarea, O. |volume=134 |pages=163–8 |year=2008 |pmid=18376043}}</ref> There have also been attempts to incorporate [[clinical decision support system]]s to further improve the quality of medicine. Prescribing diagnostic and laboratory tests is a key component of medical consultation. In the primary health care setting, tests are often ordered with preventive intentions and fulfillment of patient expectations.<ref name="MartinsAPop13">{{cite journal |title=A population-based nationwide cross-sectional study on preventive health services utilization in Portugal - What services (and frequencies) are deemed necessary by patients? |journal=PLOS One |author=Martins, C.; Azevedo, L.F.; Ribeiro, O. |volume=8 |issue=11 |pages=e81256 |year=2013 |doi=10.1371/journal.pone.0081256 |pmid=24278405 |pmc=PMC3836775}}</ref><ref name="vanBokhovenWhy06">{{cite journal |title=Why do patients want to have their blood tested? A qualitative study of patient expectations in general practice |journal=BMC Family Practice |author=van Bokhoven, M.A.; Pleunis-van Empel, M.C.; Koch, H. et al. |volume=7 |pages=75 |year=2006 |doi=10.1186/1471-2296-7-75 |pmid=17166263 |pmc=PMC1769380}}</ref> There is also great uncertainty and variability among family physicians’ ordering routines.<ref name="vanDerWeijdenUnder02">{{cite journal |title=Understanding laboratory testing in diagnostic uncertainty: A qualitative study in general practice |journal=British Journal of General Practice |author=van der Weijden, T.; van Bokhoven, M.A.; Dinant, G.J. et al. |volume=52 |issue=485 |pages=974–80 |year=2002 |pmid=12528582 |pmc=PMC1314466}}</ref><ref name="vanBokhovenExploring08">{{cite journal |title=Exploring the black box of change in improving test-ordering routines |journal=Family Practice |author=van Bokhoven, M.A.; Koch, H.; Dinant, G.J. et al. |volume=25 |issue=3 |pages=139-45 |year=2008 |doi=10.1093/fampra/cmn022 |pmid=18535302}}</ref><ref name="VerstappenVariation04">{{cite journal |title=Variation in test ordering behaviour of GPs: Professional or context-related factors? |journal=Family Practice |author=Verstappen, W.H.; ter Riet, G.; Dubois, W.I. et al. |volume=21 |issue=4 |pages=387-95 |year=2004 |doi=10.1093/fampra/cmh408 |pmid=15249527}}</ref> The effects of test ordering communication systems integrated with clinical decision support systems have been reported in various clinical practice settings. Main ''et al.'' have performed a systematic review of this topic and reported that integration of clinical decision support systems resulted in significant benefits to the prescribing process and practitioner performance outcomes in nearly two-thirds of the 24 studies that met the inclusion criteria.<ref name="MainComput10">{{cite journal |title=Computerised decision support systems in order communication for diagnostic, screening or monitoring test ordering: Systematic reviews of the effects and cost-effectiveness of systems |journal=Health Technology Assessment |author=Main, C.; Moxham, T.; Wyatt, J.C. et al. |volume=14 |issue=48 |pages=1–227 |year=2010 |doi=10.3310/hta14480 |pmid=21034668}}</ref>
To achieve better patient safety standards and improve healthcare system cost-effectiveness, there has been a worldwide effort to implement an integrated EHR system with diagnostic and [[Computerized physician order entry|laboratory test ordering communication systems]].<ref name="MoenEHealth13">{{cite journal |title=eHealth in Europe - Status and challenges |journal=Yearbook of Medical Informatics |author=Moen, A.; Hackl, W.O.; Hofdijk, J. et al. |volume=8 |pages=59–63 |year=2013 |pmid=23974549}}</ref><ref name="OlssonEuropean04">{{cite journal |title=European Commission activities in eHealth |journal=International Journal of Circumpolar Health |author=Olsson, S.; Lymberis, A.; Whitehouse, D. |volume=63 |issue=4 |pages=310-6 |year=2004 |pmid=15709306}}</ref><ref name="IakovidisEHealth08">{{cite journal |title=eHealth in Europe: From vision to reality |journal=Studies in Health Technology and Informatics |author=Iakovidis, I.; Purcarea, O. |volume=134 |pages=163–8 |year=2008 |pmid=18376043}}</ref> There have also been attempts to incorporate [[clinical decision support system]]s to further improve the quality of medicine. Prescribing diagnostic and laboratory tests is a key component of medical consultation. In the primary health care setting, tests are often ordered with preventive intentions and fulfillment of patient expectations.<ref name="MartinsAPop13">{{cite journal |title=A population-based nationwide cross-sectional study on preventive health services utilization in Portugal - What services (and frequencies) are deemed necessary by patients? |journal=PLOS One |author=Martins, C.; Azevedo, L.F.; Ribeiro, O. |volume=8 |issue=11 |pages=e81256 |year=2013 |doi=10.1371/journal.pone.0081256 |pmid=24278405 |pmc=PMC3836775}}</ref><ref name="vanBokhovenWhy06">{{cite journal |title=Why do patients want to have their blood tested? A qualitative study of patient expectations in general practice |journal=BMC Family Practice |author=van Bokhoven, M.A.; Pleunis-van Empel, M.C.; Koch, H. et al. |volume=7 |pages=75 |year=2006 |doi=10.1186/1471-2296-7-75 |pmid=17166263 |pmc=PMC1769380}}</ref> There is also great uncertainty and variability among family physicians’ ordering routines.<ref name="vanDerWeijdenUnder02">{{cite journal |title=Understanding laboratory testing in diagnostic uncertainty: A qualitative study in general practice |journal=British Journal of General Practice |author=van der Weijden, T.; van Bokhoven, M.A.; Dinant, G.J. et al. |volume=52 |issue=485 |pages=974–80 |year=2002 |pmid=12528582 |pmc=PMC1314466}}</ref><ref name="vanBokhovenExploring08">{{cite journal |title=Exploring the black box of change in improving test-ordering routines |journal=Family Practice |author=van Bokhoven, M.A.; Koch, H.; Dinant, G.J. et al. |volume=25 |issue=3 |pages=139-45 |year=2008 |doi=10.1093/fampra/cmn022 |pmid=18535302}}</ref><ref name="VerstappenVariation04">{{cite journal |title=Variation in test ordering behaviour of GPs: Professional or context-related factors? |journal=Family Practice |author=Verstappen, W.H.; ter Riet, G.; Dubois, W.I. et al. |volume=21 |issue=4 |pages=387-95 |year=2004 |doi=10.1093/fampra/cmh408 |pmid=15249527}}</ref> The effects of test ordering communication systems integrated with clinical decision support systems have been reported in various clinical practice settings. Main ''et al.'' have performed a systematic review of this topic and reported that integration of clinical decision support systems resulted in significant benefits to the prescribing process and practitioner performance outcomes in nearly two-thirds of the 24 studies that met the inclusion criteria.<ref name="MainComput10">{{cite journal |title=Computerised decision support systems in order communication for diagnostic, screening or monitoring test ordering: Systematic reviews of the effects and cost-effectiveness of systems |journal=Health Technology Assessment |author=Main, C.; Moxham, T.; Wyatt, J.C. et al. |volume=14 |issue=48 |pages=1–227 |year=2010 |doi=10.3310/hta14480 |pmid=21034668}}</ref>
In Portugal, the use of EHR software with a diagnostic and laboratory test order communication system has been mandatory since September 2011. Most of the primary healthcare centers use software called ''Sistema de Apoio ao Médico'' (Physician’s Support System [SAM]). In the module used to order diagnostic and laboratory tests, physicians access a searchable test menu by two possible strategies: 1) typing the test name in a search box or 2) browsing by a shortcut menu structure (Fig. 1). Different menus are available for most areas of medicine, including basic, allergology, andrology, cardiovascular, central nervous system, digestive, dosing, endocrinology, gynecology, hematology, infectious diseases, nephrology, obstetrics, oncology, otorhinolaryngology, osteoarticular, preoperative, respiratory, rheumatology, and urology. Under each menu there is a set of specific lab tests. Physicians can choose one or more tests by double-clicking each test or can choose the entire set by double-clicking on the shortcut menu’s title. For example, the basic menu is composed of uric acid, total cholesterol, creatinine, gamma-glutamyl transferase, glucose, hemogram, serum protein electrophoresis, aspartate aminotransferase, urine type 2, sedimentation rate, electrocardiogram, and lung x-ray tests.
[[File:Fig1 Martins BMCMedInfoDecMak2017 17-1.gif|472px]]
{{clear}}
{|
| STYLE="vertical-align:top;"|
{| border="0" cellpadding="5" cellspacing="0" width="472px"
|-
  | style="background-color:white; padding-left:10px; padding-right:10px;"| <blockquote>'''Figure 1.''' Usual ordering communication system: the basic shortcut menu</blockquote>
|-
|}
|}
If a physician would like to choose only hemogram and glucose, they must double-click on each test. However, if they would like the entire basic set of tests, they must double-click on the basic menu title. Our research team suspected that this basic shortcut menu is often selected during routine consultations in which patients ask for routine check-ups without specific reason. As we have shown in a previous study, there is a high prevalence of Portuguese adults (99.2%) that believe they should have routine blood and urine tests annually.<ref name="MartinsAPop13" /> This statistic demonstrates the importance of examining the effectiveness and efficiency of this basic sub-menu.
Through a randomized controlled trial, the primary aim of the present study was to compare the effects of modifying the EHR ordering communication system (modified SAM) by changing the basic shortcut menu and adding a clinical decision support system based on the integration of the United States Preventive Services Task Force (USPSTF) recommendations. After the last primary healthcare reform in Portugal, primary healthcare centers have been divided into healthcare center groups.<ref name="daCostaPereiraEHealth10">{{cite web |url=http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.476.4177&rep=rep1&type=pdf |format=PDF |title=eHealth Strategies - Country Brief: Portugal |author=da Costa Pereira, A.; Giest, S.; Dumortier, J.; Artmann, J. |publisher=European Commission |date=October 2010}}</ref> In a healthcare center group, the informatics network is linked through servers that may serve more than one healthcare center. Creating a modified version of the SAM software requires it to be installed at a server level, which determines that all physicians at all healthcare centers served by that server will receive the same version of the software. For this reason, it was not possible to randomize the study at the physician level. Rather, we had to randomize the servers at the healthcare center group level.
==Methods==


==References==
==References==

Revision as of 20:01, 10 April 2017

Full article title The effect of a test ordering software intervention on the prescription of unnecessary laboratory tests - A randomized controlled trial
Journal BMC Medical Informatics and Decision Making
Author(s) Martins, C.M.; da Costa Teixeira, A.S.; de Azevedo. L.F.; Sá, L.M.; Santos, P.A.; do Couto, M.L.; da Costa Pereira, A.M.;
Hespanhol, A.A.; da Costa Santos, C.M.
Author affiliation(s) University of Porto
Primary contact E-mail: carlosmartins20 at gmail dot com
Year published 2017
Volume and issue 17 (1)
Page(s) 20
DOI 10.1186/s12911-017-0416-6
ISSN 1472-6947
Distribution license Creative Commons Attribution 4.0 International
Website https://bmcmedinformdecismak.biomedcentral.com/articles/10.1186/s12911-017-0416-6
Download https://bmcmedinformdecismak.biomedcentral.com/track/pdf/10.1186/s12911-017-0416-6 (PDF)
This article should not be considered complete until this message box has been removed. This is a work in progress.

Abstract

Background: The way electronic health record and laboratory test ordering system software is designed may influence physicians’ prescription. A randomized controlled trial was performed to measure the impact of a diagnostic and laboratory tests ordering system software modification.

Methods: Participants were family physicians working and prescribing diagnostic and laboratory tests.

The intervention group had modified software with basic shortcut menu changes, where some tests were withdrawn or added, and with the implementation of an evidence-based clinical decision support system based on United States Preventive Services Task Force (USPSTF) recommendations. This intervention group was compared with typically used software (control group).

The outcomes were the number of tests prescribed from those: withdrawn from the basic menu; added to the basic menu; marked with green dots (USPSTF’s grade A and B); and marked with red dots (USPSTF’s grade D).

Results: Comparing the monthly average number of tests prescribed before and after the software modification, from those tests that were withdrawn from the basic menu, the control group prescribed 33.8 tests per 100 consultations before and 30.8 after (p = 0075); the intervention group prescribed 31.3 before and 13.9 after (p < 0001). Comparing the tests prescribed between both groups during the intervention, from those tests that were withdrawn from the basic menu, the intervention group prescribed a monthly average of 14.0 vs. 29.3 tests per 100 consultations in the control group (p < 0.001). From those tests that are USPSTF’s grade A and B, the intervention group prescribed 66.8 vs. 74.1 tests per 100 consultations in the control group (p = 0.070). From those tests categorized as USPSTF grade D, the intervention group prescribed an average of 9.8 vs. 11.8 tests per 100 consultations in the control group (p = 0.003).

Conclusions: Removing unnecessary tests from a quick shortcut menu of the diagnosis and laboratory tests ordering system had a significant impact and reduced unnecessary prescription of tests.

The fact that it was not possible to perform the randomization at the family physicians’ level, but only on the computer servers is a limitation of our study. Future research should assess the impact of different test ordering systems during longer periods.

Trial registration: ISRCTN45427977, May 1st 2014 (retrospectively registered).

Keywords: Preventive health services, primary health care, evidence-based practice, decision support systems, clinical decision making, computer-assisted

Background

Informatics has undoubtedly changed the way societies live, socialize, learn, work, and deal with healthcare. We now live in a period of increasing concern about the excessive presence of medicine in our lives.[1][2][3] When inefficient software is combined with a non-evidence-based medical practice, there is the risk of patient harm, significant impact to quality of life, and damage to the healthcare system due to unnecessary costs.

The implementation of electronic health records (EHRs) has both potential benefits and drawbacks.[4] Among the benefits, the prevention of medical errors and the promotion of patient safety has often been mentioned and confirmed in clinical practice.[4][5][6] Despite the positive effects of EHR implementation in clinical practice, a range of barriers faced by physicians has been identified. These barriers may include technical and financial aspects, time, psychological, social, legal, and organizational changes to the process.[7] After having removed the first barriers to EHR implementation, it is now time to implement continuing improvement and development of the available tools and to incorporate the scientific evidence obtained to this point.[4][8][9][10]

To achieve better patient safety standards and improve healthcare system cost-effectiveness, there has been a worldwide effort to implement an integrated EHR system with diagnostic and laboratory test ordering communication systems.[11][12][13] There have also been attempts to incorporate clinical decision support systems to further improve the quality of medicine. Prescribing diagnostic and laboratory tests is a key component of medical consultation. In the primary health care setting, tests are often ordered with preventive intentions and fulfillment of patient expectations.[14][15] There is also great uncertainty and variability among family physicians’ ordering routines.[16][17][18] The effects of test ordering communication systems integrated with clinical decision support systems have been reported in various clinical practice settings. Main et al. have performed a systematic review of this topic and reported that integration of clinical decision support systems resulted in significant benefits to the prescribing process and practitioner performance outcomes in nearly two-thirds of the 24 studies that met the inclusion criteria.[19]

In Portugal, the use of EHR software with a diagnostic and laboratory test order communication system has been mandatory since September 2011. Most of the primary healthcare centers use software called Sistema de Apoio ao Médico (Physician’s Support System [SAM]). In the module used to order diagnostic and laboratory tests, physicians access a searchable test menu by two possible strategies: 1) typing the test name in a search box or 2) browsing by a shortcut menu structure (Fig. 1). Different menus are available for most areas of medicine, including basic, allergology, andrology, cardiovascular, central nervous system, digestive, dosing, endocrinology, gynecology, hematology, infectious diseases, nephrology, obstetrics, oncology, otorhinolaryngology, osteoarticular, preoperative, respiratory, rheumatology, and urology. Under each menu there is a set of specific lab tests. Physicians can choose one or more tests by double-clicking each test or can choose the entire set by double-clicking on the shortcut menu’s title. For example, the basic menu is composed of uric acid, total cholesterol, creatinine, gamma-glutamyl transferase, glucose, hemogram, serum protein electrophoresis, aspartate aminotransferase, urine type 2, sedimentation rate, electrocardiogram, and lung x-ray tests.


Fig1 Martins BMCMedInfoDecMak2017 17-1.gif

Figure 1. Usual ordering communication system: the basic shortcut menu

If a physician would like to choose only hemogram and glucose, they must double-click on each test. However, if they would like the entire basic set of tests, they must double-click on the basic menu title. Our research team suspected that this basic shortcut menu is often selected during routine consultations in which patients ask for routine check-ups without specific reason. As we have shown in a previous study, there is a high prevalence of Portuguese adults (99.2%) that believe they should have routine blood and urine tests annually.[14] This statistic demonstrates the importance of examining the effectiveness and efficiency of this basic sub-menu.

Through a randomized controlled trial, the primary aim of the present study was to compare the effects of modifying the EHR ordering communication system (modified SAM) by changing the basic shortcut menu and adding a clinical decision support system based on the integration of the United States Preventive Services Task Force (USPSTF) recommendations. After the last primary healthcare reform in Portugal, primary healthcare centers have been divided into healthcare center groups.[20] In a healthcare center group, the informatics network is linked through servers that may serve more than one healthcare center. Creating a modified version of the SAM software requires it to be installed at a server level, which determines that all physicians at all healthcare centers served by that server will receive the same version of the software. For this reason, it was not possible to randomize the study at the physician level. Rather, we had to randomize the servers at the healthcare center group level.

Methods

References

  1. ↑ Glasziou, P.; Moynihan, R.; Richards, T. et al. (2013). "Too much medicine; too little care". BMJ 347: f4247. doi:10.1136/bmj.f4247. PMID 23820022. 
  2. ↑ Moynihan, R.; Doust, J.; Henry, D. (2012). "Preventing overdiagnosis: How to stop harming the healthy". BMJ 344: e3502. doi:10.1136/bmj.e3502. PMID 22645185. 
  3. ↑ Getz, L.; Sigurdsson, J.A.; Hetlevik, I. (2003). "Is opportunistic disease prevention in the consultation ethically justifiable?". BMJ 327 (7413): 498–500. doi:10.1136/bmj.327.7413.498. PMC PMC188390. PMID 12946974. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC188390. 
  4. ↑ 4.0 4.1 4.2 Raposo, V.L. (2015). "Electronic health records: Is it a risk worth taking in healthcare delivery?". GMS Health Technology Assessment 11: Doc02. doi:10.3205/hta000123. PMC PMC4677576. PMID 26693253. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4677576. 
  5. ↑ Ben-Assuli, O. (2015). "Electronic health records, adoption, quality of care, legal and privacy issues and their implementation in emergency departments". Health Policy 119 (3): 287–97. doi:10.1016/j.healthpol.2014.11.014. PMID 25483873. 
  6. ↑ Ben-Assuli, O.; Leshno, M. (2016). "Assessing electronic health record systems in emergency departments: Using a decision analytic Bayesian model". Health Informatics Journal 22 (3): 712–29. doi:10.1177/1460458215584203. PMID 26033468. 
  7. ↑ Boonstra, A.; Broekhuis, M. (2010). "Barriers to the acceptance of electronic medical records by physicians from systematic review to taxonomy and interventions". BMC Health Services Research 10: 231. doi:10.1186/1472-6963-10-231. PMC PMC2924334. PMID 20691097. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2924334. 
  8. ↑ Roukema, J.; Los, R.K.; Bleeker, S.E. (2006). "Paper versus computer: Feasibility of an electronic medical record in general pediatrics". Pediatrics 117 (1): 15–21. doi:10.1542/peds.2004-2741. PMID 16396855. 
  9. ↑ Bowman, S. (2013). "Impact of electronic health record systems on information integrity: Quality and safety implications". Perspectives in Health Information Management 10: 1c. PMC PMC3797550. PMID 24159271. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3797550. 
  10. ↑ Nguyen, L.; Bellucci, E.; Nguyen, L.T. (2014). "Electronic health records implementation: An evaluation of information system impact and contingency factors". International Journal of Medical Informatics 83 (11): 779–96. doi:10.1016/j.ijmedinf.2014.06.011. PMID 25085286. 
  11. ↑ Moen, A.; Hackl, W.O.; Hofdijk, J. et al. (2013). "eHealth in Europe - Status and challenges". Yearbook of Medical Informatics 8: 59–63. PMID 23974549. 
  12. ↑ Olsson, S.; Lymberis, A.; Whitehouse, D. (2004). "European Commission activities in eHealth". International Journal of Circumpolar Health 63 (4): 310-6. PMID 15709306. 
  13. ↑ Iakovidis, I.; Purcarea, O. (2008). "eHealth in Europe: From vision to reality". Studies in Health Technology and Informatics 134: 163–8. PMID 18376043. 
  14. ↑ 14.0 14.1 Martins, C.; Azevedo, L.F.; Ribeiro, O. (2013). "A population-based nationwide cross-sectional study on preventive health services utilization in Portugal - What services (and frequencies) are deemed necessary by patients?". PLOS One 8 (11): e81256. doi:10.1371/journal.pone.0081256. PMC PMC3836775. PMID 24278405. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3836775. 
  15. ↑ van Bokhoven, M.A.; Pleunis-van Empel, M.C.; Koch, H. et al. (2006). "Why do patients want to have their blood tested? A qualitative study of patient expectations in general practice". BMC Family Practice 7: 75. doi:10.1186/1471-2296-7-75. PMC PMC1769380. PMID 17166263. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1769380. 
  16. ↑ van der Weijden, T.; van Bokhoven, M.A.; Dinant, G.J. et al. (2002). "Understanding laboratory testing in diagnostic uncertainty: A qualitative study in general practice". British Journal of General Practice 52 (485): 974–80. PMC PMC1314466. PMID 12528582. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1314466. 
  17. ↑ van Bokhoven, M.A.; Koch, H.; Dinant, G.J. et al. (2008). "Exploring the black box of change in improving test-ordering routines". Family Practice 25 (3): 139-45. doi:10.1093/fampra/cmn022. PMID 18535302. 
  18. ↑ Verstappen, W.H.; ter Riet, G.; Dubois, W.I. et al. (2004). "Variation in test ordering behaviour of GPs: Professional or context-related factors?". Family Practice 21 (4): 387-95. doi:10.1093/fampra/cmh408. PMID 15249527. 
  19. ↑ Main, C.; Moxham, T.; Wyatt, J.C. et al. (2010). "Computerised decision support systems in order communication for diagnostic, screening or monitoring test ordering: Systematic reviews of the effects and cost-effectiveness of systems". Health Technology Assessment 14 (48): 1–227. doi:10.3310/hta14480. PMID 21034668. 
  20. ↑ da Costa Pereira, A.; Giest, S.; Dumortier, J.; Artmann, J. (October 2010). "eHealth Strategies - Country Brief: Portugal" (PDF). European Commission. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.476.4177&rep=rep1&type=pdf. 

Notes

This presentation is faithful to the original, with only a few minor changes to presentation. In some cases important information was missing from the references, and that information was added. A significant amount of grammar edits were made to make the document more readable.

Retrieved from "https://www.limswiki.org/index.php?title=Journal:The_effect_of_a_test_ordering_software_intervention_on_the_prescription_of_unnecessary_laboratory_tests_-_A_randomized_controlled_trial&oldid=29884"