Journal:Beyond information retrieval and electronic health record use: Competencies in clinical informatics for medical education
|Full article title||Beyond information retrieval and electronic health record use: Competencies in clinical informatics for medical education|
|Journal||Advances in Medical Education and Practice|
|Author(s)||Hersh, William R.; Gorman, Paul N.; Biagioli, Frances E.; Mohan, Vishnu; Gold, Jeffrey A.; Mejicano, George C.|
|Author affiliation(s)||Oregon Health & Science University, Portland, OR, USA|
|Volume and issue||2014 (5)|
|Distribution license||Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported|
Physicians in the 21st century will increasingly interact in diverse ways with information systems, requiring competence in many aspects of clinical informatics. In recent years, many medical school curricula have added content in information retrieval (search) and basic use of the electronic health record. However, this omits the growing number of other ways that physicians are interacting with information that includes activities such as clinical decision support, quality measurement and improvement, personal health records, telemedicine, and personalized medicine. We describe a process whereby six faculty members representing different perspectives came together to define competencies in clinical informatics for a curriculum transformation process occurring at Oregon Health & Science University. From the broad competencies, we also developed specific learning objectives and milestones, an implementation schedule, and mapping to general competency domains. We present our work to encourage debate and refinement as well as facilitate evaluation in this area.
Keywords: curriculum transformation, clinical decision support, patient safety, health care quality, patient engagement
Physicians and medical students have been using health information technology (HIT) for decades. During this time, the role of HIT has changed dramatically from a useful tool for data access and occasional information retrieval to a ubiquitous presence that permeates health care and medical practice in myriad ways. As the use of HIT has accelerated and the underlying science of biomedical and health informatics has advanced, medical education has lagged behind, leaving physicians and students alike to learn largely on their own how to make use of these tools. While some medical schools have introduced limited aspects of biomedical informatics into their curricula, most of this is focused on training to perform basic tasks such as accessing knowledge resources or basic use of an electronic health record (EHR).
Twenty-first century clinicians face a much more digital world than their predecessors. The quantity of biomedical knowledge continues to expand, with an attendant increase in the primary scientific literature. Secondary sources that summarize this information proliferate as well, not only for use by clinicians but also by patients and consumers. Programs such as the “meaningful use” financial incentives of the Health Information for Technology and Clinical Health Act require comprehensive adoption and use of the EHR. Patients, especially the aging “baby boomer” generation, want to interact with the health care system the same way they interact with airlines, banks, and retailers, ie, through digital means using technologies such as the personal health record. Patients, payers, and purchasers demand more accountability in health care quality, safety, and cost, leading to an expectation of measurement and reporting of quality of care as a routine part of participation in new delivery mechanisms such as primary care medical homes and accountable care organizations. At the same time, telemedicine applications extend the reach of health care systems and clinicians in both rural and urban settings. Ideally, for all of these applications, patient data must move readily across organizational boundaries via health information exchange, while privacy and confidentiality are protected. The growing quantity of clinical and administrative data in these systems also affords an opportunity for advanced analysis that can enable better deployment of resources and coordination of care, facilitation of personalized and precision medicine, and advancement of clinical and translational research. Together, these advances are moving health care toward the global vision of the learning health system put forth by the Institute of Medicine (IOM).
The importance of these developments is further evidenced by the recent establishment of a new medical subspecialty of clinical informatics. Practicing physicians are now beginning to become board-certified in this new subspecialty, with fellowship programs accredited by the Accreditation Council for Graduate Medical Education (ACGME) to soon be established. This underscores the need for medical students to be introduced to the concepts and competencies of this new medical subspecialty as part of the undergraduate curriculum.
In this paper, we describe our work to expand the undergraduate medical curriculum to include a comprehensive set of competencies in clinical informatics. We go beyond the usual focus on information retrieval from knowledge sources and data access through the EHR, aiming to address the expanded and diverse roles of HIT in contemporary health care delivery, personal health, public health, and clinical and translational research. We describe the development of a novel biomedical informatics curriculum that is being implemented within a comprehensive transformation of the undergraduate curriculum at our institution. Our future plans include evaluation of the impact of this work, and we encourage others to participate in this evaluation and refinement process.
Previous work developing competencies in informatics
The need to include biomedical informatics in medical education has been recognized in previous publications of competencies, curricula, and other learning materials for informatics. One early, high-profile resource was part of the Medical Student Objectives Project of the Association of American Medical Colleges, which framed proposed objectives in terms of five key roles of physicians: clinicians, life-long learners, educators, researchers, and managers. Although current medical students believe these objectives are important to their education, uptake of these objectives has been modest. More recently, the Alliance for Clinical Education issued a policy statement calling for competencies related to the EHR, although also noting that many challenges to implementing such competencies exist. Others have also noted the challenges to education about the EHR, citing problems analogous to those of clinicians incorporating EHRs into practice, such as integration into workflows, and concerns about unanticipated and undesirable impacts. It is clear, however, that competencies in the use of the EHR are aligned with more general physician educational competencies, and that informatics curricula can be incorporated into undergraduate medical training and evaluated.
In 2012, the Oregon Health & Science University, School of Medicine began a comprehensive transformation of the undergraduate medical curriculum for the 21st century. From the outset, the Dean and school leadership recognized the importance of informatics to physicians and health care in the future. With participation and support from the Senior Associate Dean for Education (GM), a team of five physician-faculty was convened to lead this effort, including the following:
- An informatician involved nationally in informatics education leadership and development of the new clinical informatics subspecialty (WH)
- An internist-informatician who is chair of the medical school curriculum committee and actively engaged in the new curriculum development team (PG)
- A family physician and innovator in using simulation for teaching and assessing EHR use by medical students (FB)
- A critical care physician investigator using EHR simulation to assess residents’ ability to detect important patterns in critical care data (JG)
- An informatician and former internal medicine residency associate program director also involved in the previous two above activities (VM)
Using an exploratory qualitative methodology based on action research and working with the curriculum transformation leadership, we developed a broad set of competencies in informatics for medical education. With each competency, we developed a set of learning objectives, a designation of when it would be most appropriate to introduce the concept during the medical school curriculum, and how each mapped to the broader ACGME competencies.
Developing competencies and learning objectives
Before beginning to identify the competencies for student learning, we needed to develop the rationale for including these topics in the curriculum. This led us to map the competencies to the six ACGME core competency domains and develop explicit learning objectives within each competency, followed by their implementation in the emerging curriculum. Discerning the rationale for our competencies was a narrative process, with their implementation enumerating the specific areas deemed important to incorporate into the medical school curriculum, as it was being concurrently designed.
Our first step was to provide the rationale to include various topic areas in the curriculum. A first critical concept we agreed upon was that informatics is not the same as computer literacy. Computer literacy is one of many requirements to use informatics successfully, but knowing how to use a computing device (personal computer, tablet, or smartphone) is not the same as having skills in informatics, ie, using that device to improve health, health care delivery, public health, or research.
We concurred that one fundamental skill currently being taught in many medical schools for 21st century clinicians was something taught in many medical schools since the late 20th century, namely, how to find information to apply to patient care. The needed informatics skills start, and do not stop, with the knowledge of how to enter simple queries into common general (eg, Google™, http://www.google.com) or medical (eg, PubMed, http://www.pubmed.gov) search engines. Clinicians must also know how to formulate a clinical question as an answerable one, and then be able to select the appropriate resource and make optimal use of it. This includes knowing what content is in different search systems. He or she must also know about specialized resources such as the Agency for Healthcare Research and Quality Guidelines Clearinghouse (http://www.guideline.gov) and the Centers for Disease Control and Prevention travel site (http://wwwnc.cdc.gov/travel). Clinicians should also have an understanding of the major commercial publishers as well as what professional societies offer. Once clinicians know how to use a search site, they must be able to phrase an appropriate query and know how to maximize the search using advanced query features. To take maximum advantage of available knowledge resources, it is necessary to understand the advanced search features that are available, whether in general web resources such as Google or dedicated biomedical knowledge resources such as PubMed.
Once information is retrieved, clinicians must know how to critically appraise the information retrieved and decide, if warranted, to apply it to the patient or population. As with searching, the type of appraisal varies with the search engine used. With output from general search engines like Google, the trustworthiness of the information must be assessed. Clinicians must also know how to make optimal use of patient data and information. They must know how to use informatics to achieve the triple aim of better health, better care, and lower cost. An appropriate framework for applying this emanates from the IOM concept of the “learning health system”, which presents a compelling vision for a health care system that is patient-centered, population-based, and promotes learning from data.
Being able to use data and information also means understanding that the patient health record is more than “charting”, and that its value goes beyond being able to read it. Clinicians must be facile with all aspects of the EHR, able to easily move from one vendor system to another and to understand the critical role of health information exchange in making any single record as complete as possible. This issue is magnified by the increasing amount of data the clinicians must view and manage. Further, as the health care focus moves from one of quantity to one of value, clinicians must pay attention to quality, patient safety, and cost. This requires coordination of care, in not just providing medical procedures, nursing interventions, therapies, etc in isolation, but also teamwork and communication.
From this vantage, the health record is no longer a passive collection of information used mainly to justify billing. Rather, it is a source of data, organized into coherent information that allows the health care team to deliver the best, safest, and most cost-effective care. This means that the 21st century clinician must have a basic understanding of informatics issues, such as capturing data that is correct and complete as well as consistent in its expression. He or she must be able to work in partnership with informatics professionals to achieve what we know is so critical in the application of informatics: adhering to standards, achieving system interoperability, appropriately and optimally implementing clinical decision support, and maintaining security to assure privacy and confidentiality.
Clinicians also require understanding of areas of quality measurement and improvement, especially as it applies to their direct practice. The clinician must understand concepts of health care quality and how the EHR and other sources of data are increasingly used to measure it. They must also know how to work with clinical leaders to select, implement, and improve quality measures. This should be introduced as part of the larger learning health system. This discussion should also include the related issues of patient safety and medical errors, including safety of HIT.
Patient engagement is another component for safe, effective, and coordinated care. Not only does patient engagement improve health care, but patients, including Internet-savvy baby boomers, will increasingly demand it. Patients want health care that adapts the online features of other modern industries, allowing them the ability to view their own data and interact with their clinicians and health care system (eg, online scheduling of appointments, prescription refills, and even consultations that are appropriate via the Internet or other electronic media). These will likely take the form of a personal health record, accessible from a patient portal that allows access to all information, not just that from the system of the provider organization. Because of this natural evolution of health care, clinicians must also have knowledge and understanding of the appropriate use of telemedicine and telehealth (such as e-visits), both for remote locations and as a convenient option locally in patients’ homes and other settings.
Complementary to the patient-centric view, clinicians must also understand population-based care and the informatics underlying it. The clinician and their team care for populations of patients; they must be able to view their care needs and results across their patient population. When a new test or treatment is determined to be highly effective, the clinician must be able to quickly identify patients who are candidates for it. They must also be able to identify outliers in their populations who require intervention, such as those with excessively high blood pressure or blood sugar, missed appointments or screen tests, or those at risk for hospital (re)admission.
Another area where 21st century clinicians must understand the key issues is in bioinformatics, especially as it relates to personalized medicine. Modern clinicians need not understand complicated gene sequencing algorithms, but they should have an understanding of how genomics and related areas are transforming our understanding of maintaining health and diagnosing and treating disease. Now that the vision of personalized medicine is coming to fruition, clinicians will increasingly rely on the help of decision support and other tools for assistance in applying it to individual patients. He or she should have a basic understanding of genome-wide association studies and their ramifications.
Clinicians must also understand the strengths and limitations of clinical and translational research. They must understand the differences between and the value of experimental and observational studies. Ideally, students will have participated in research while in their training. But even if not, they should understand issues such as data quality, study design, and the limitations that come from the sharp focus perspective of a clinical study. Students should participate in the learning health system laid out in the vision of groups like the IOM.
Implementing competencies and objectives
Based on the above narrative, our group of diverse clinician-educators developed through an iterative process a set of 13 competencies in clinical informatics (Table 1). Each competency was mapped to one or more of the six ACGME general competency domains. We then developed more detailed learning objectives and milestones within each competency. We also categorized each learning objective for its presence in the early (at the beginning), middle (during the preclinical portion), or late (during clinical experiences) portion of the curriculum.
Table 1 Competencies in clinical informatics and specific learning objective/milestone within each
Notes: All learning objectives/milestones are designated for time to start (early: start of medical school; mid: later in preclinical years; and late: during clinical experiences); Each is also mapped into one or more of the ACGME core competencies: patient care and procedural skills (PCPS), medical knowledge (MK), practice-based learning and improvement (PBLI), interpersonal and communication skills (ICS), professionalism (Prof), and systems-based practice (SBP).Abbreviations: ACGME, Accreditation Council for Graduate Medical Education; CDS, clinical decision support; HIE, health information exchange; HIPAA, Health Insurance Portability and Accountability Act.
These learning objectives are now being woven into Oregon Health & Science University’s new curriculum (Table 2). As seen in Table 2, we have designated where the specific learning activities will be placed in the major portions of the curriculum from orientation through preclinical sciences, clinical experiences, and intersession in the fourth year.
Table 2 Informatics in various portions of curriculum in the preclinical (orientation, fundamentals, and preclinical sciences) and clinical (clinical experiences and intersession) years
Abbreviations: CDS, clinical decision support; DDI, drug–drug interaction; EBM, evidence-based medicine; EHR, electronic health record; EKG, electrocardiogram; HIE, health information exchange; HIPAA, Health Insurance Portability and Accountability Act; IT, information technology; OHSU, Oregon Health & Science University; PHR, personal health record; VA, Veterans Affairs.
Conclusion and future directions
Our analysis has shown there are a substantial number of informatics competencies and a large body of associated knowledge that the 21st century clinician needs to learn and apply. From a pedagogical standpoint, there are also issues in how to organize, deliver, and assess this content. Certainly, one approach is to provide this content as a separate course, isolated from the rest of the curriculum. However, a better approach would be to tightly and comprehensively integrate informatics concepts longitudinally into the learning curriculum since clinical informatics is emerging as a core competency of medical practice, applicable in all basic science disciplines and clinical specialties.
A next major step for this work will be to develop evaluation activities for the competencies and learning activities. These will vary based on institutional factors (class size, whether students are asynchronous, etc), faculty preference, and funding limitations (desire to use simulation for many things, but cost may be prohibitive). As such, different learning activities will require different evaluation methods. For example, the competency “Provide care using population health management approaches” might be best measured by having students describe or select effective system change options based on immunizations rates of the clinic to which they are assigned, while the competency “Provide clinical care via telemedicine and refer those for whom it is necessary” might best be measured with an objective structured clinical examination simulation.
The presence of these competencies also indicates a need for educators who are specialists in informatics to (collaboratively with clinical educators) design the learning and deliver learning experiences that are appropriate for lecture, group discussions, self-paced and self-directed methods, and other settings. But informatics is one of those topics that is best infused throughout the curriculum, especially in clinical settings where it is being used.
There are a number of future steps for this work. First, we recognize that this set of competencies represents the views of one medical school’s informatics faculty and must be validated with informatics experts from other institutions. Second, we must evaluate our own implementation of this curriculum to determine how these competencies are delivered to a medical student audience. Finally, we must evaluate this entire process with students from our institution and others to determine which competencies and learning experiences are most valuable for them in their future clinical practice.
Dr Biagioli was supported in part by NIH Grant 1R25CA158571. Drs Hersh and Mejicano were supported in part by the Accelerating Change in Medical Education grant of the American Medical Association.
The authors report no conflicts of interest in this work.
- Bastian, H.; Glasziou, P.; Chalmers, I. (2010). "Seventy-five trials and eleven systematic reviews a day: how will we ever keep up?". PLoS Medicine 7 (9): e1000326. doi:10.1371/journal.pmed.1000326. PMC PMC2943439. PMID 20877712. http://www.pubmedcentral.nih.gov/articlerender.fcgi?tool=pmcentrez&artid=PMC2943439.
- Blumenthal, D. (2011). "Wiring the health system – origins and provisions of a new federal program". The New England Journal of Medicine 365 (24): 2323–2329. doi:10.1056/NEJMsr1110507. PMID 22168647.
- Blumenthal, D. (2011). "Implementation of the federal health information technology initiative". The New England Journal of Medicine 365 (25): 2426–2431. doi:10.1056/NEJMsr1112158. PMID 22187990.
- Miller, H.; Yasnoff, W.; Burde, H. (2009). Personal Health Records: The Essential Missing Element in 21st Century Healthcare. Chicago, IL: Healthcare Information and Management Systems Society. pp. 178. ISBN 9780980069761.
- Berwick, D.M.; Nolan, T.W.; Whittington, J. (2008). "The triple aim: care, health, and cost". Health Affairs 27 (3): 759–769. doi:10.1377/hlthaff.27.3.759. PMID 18474969.
- Kuperman, G.J. (2011). "Health-information exchange: why are we doing it, and what are we doing?". Journal of the American Medical Informatics Association 18 (5): 678–682. doi:10.1136/amiajnl-2010-000021. PMC PMC3168299. PMID 21676940. http://www.pubmedcentral.nih.gov/articlerender.fcgi?tool=pmcentrez&artid=PMC3168299.
- Safran, C.; Bloomrosen, M.; Hammond, W.E.; et al. (2007). "Toward a national framework for the secondary use of health data: an American Medical Informatics Association White Paper". Journal of the American Medical Informatics Association 14 (1): 1–9. doi:10.1197/jamia.M2273. PMC PMC2329823. PMID 17077452. http://www.pubmedcentral.nih.gov/articlerender.fcgi?tool=pmcentrez&artid=PMC2329823.
- Friedman, C.P.; Wong, A.K.; Blumenthal, D. (2010). "Achieving a nationwide learning health system". Science Translation Medicine 2 (57): 57cm29. doi:10.1126/scitranslmed.3001456.
- Smith, M.; Saunders, R.; Stuckhardt, L.; McGinnis, J.M., ed. (2012). Best Care at Lower Cost: The Path to Continuously Learning Health Care in America. Washington, DC: National Academies Press. pp. 416. ISBN 9780309260732.
- Lehmann, C.U.; Shorte, V.; Gundlapalli, A.V. (2013). "Clinical informatics sub-specialty board certification". Pediatrics in Review 34 (11): 525–530. doi:10.1542/pir.34-11-525. PMID 24187144.
- The Informatics Panel and the Population Health Perspective Panel (1999). "Contemporary issues in medicine – medical informatics and population health: report II of the Medical School Objectives Project". Academic Medicine 74 (2): 130–141. http://journals.lww.com/academicmedicine/Abstract/1999/02000/Contemporary_issues_in_medicine_medical.12.aspx.
- Beaudoin, D.; Richardson, S.J.; Sheng, X.; Mitchell, J. (2013). "Medical students’ perspectives on biomedical informatics learning objectives". International Journal of Medical Education (4): 1–8. doi:10.5116/ijme.50ce.316b. PMC PMC4205536. http://www.pubmedcentral.nih.gov/articlerender.fcgi?tool=pmcentrez&artid=PMC4205536.
- McGowan, J.; Passiment, M.; Hoffman, H. (2007). "Educating medical students as competent users of health information technologies: the MSOP data". Studies in Health Technology and Informatics 129 (Pt. 2): 1414-8. PMID 17911947.
- Hammoud, M.M.; Dalymple, J.L.; Christner, J.G.; et al. (2012). "Medical student documentation in electronic health records: a collaborative statement from the Alliance for Clinical Education". Teaching and Learning in Medicine 24 (3): 257–266. doi:10.1080/10401334.2012.692284. PMID 22775791.
- Hammoud, M.M.; Margo, K.; Christner, J.G.;, Fisher, J.; Fischer, S.H.; Pangaro, L.N. (2012). "Opportunities and challenges in integrating electronic health records into undergraduate medical education: a national survey of clerkship directors". Teaching and Learning in Medicine 24 (3): 219–224. doi:10.1080/10401334.2012.692267. PMID 22775785.
- Ellaway, R.H.; Graves, L.; Greene, P.S. (2013). "Medical education in an electronic health record-mediated world". Medical Teacher 35 (4): 282–286. doi:10.3109/0142159X.2013.773396. PMID 23464893.
- Tierney, M.J.; Pageler, N.M.; Kahana, M.; Pantaleoni, J.L.; Longhurst, C.A. (2013). "Medical education in the electronic medical record (EMR) era: benefits, challenges, and future directions". Academic Medicine 88 (6): 748–752. doi:10.1097/ACM.0b013e3182905ceb. PMID 23619078.
- Pageler, N.M.; Friedman, C.P.; Longhurst, C.A. (2013). "Refocusing medical education in the EMR era". Journal of the American Medical Association 310 (21): 2249–2250. doi:10.1001/jama.2013.282326. PMID 24302083.
- Silverman, H.; Cohen, T.; Fridsma, D. (2012). "The evolution of a novel biomedical informatics curriculum for medical students". Academic Medicine 87 (1): 84–90. doi:10.1097/ACM.0b013e31823a599e. PMID 22104049.
- Milano, C.E.; Hardman, J.A.; Plesiu, A.; Rdesinski, R.E.; Biagioli, F.E. (2014). "Simulated electronic health record (Sim-EHR) curriculum: teaching EHR skills and use of the EHR for disease management and prevention". Academic Medicine 89 (3): 399–403. doi:10.1097/ACM.0000000000000149. PMC PMC4035239. PMID 24448035. http://www.pubmedcentral.nih.gov/articlerender.fcgi?tool=pmcentrez&artid=PMC4035239.
- Reis, S.; Sagi, D.; Eisenberg, O.; et al. (2013). "The impact of residents’ training in Electronic Medical Record (EMR) use on their competence: report of a pragmatic trial". Patient Education and Counseling 93 (3): 515–521. doi:10.1016/j.pec.2013.08.007. PMID 24011647.
- Englander, R.; Cameron, T.; Ballard, A.J.; Dodge, J.; Bull, J.; Aschenbrener, C.A. (2013). "Toward a common taxonomy of competency domains for the health professions and competencies for physicians". Academic Medicine 88 (8): 1088–1094. doi:10.1097/ACM.0b013e31829a3b2b. PMID 23807109.
- Manor-Shulman, O.; Beyene, J.; Frndova, H.; Parshuram, C.S. (2008). "Quantifying the volume of documented clinical information in critical illness". Journal of Critical Care 23 (2): 245–250. doi:10.1016/j.jcrc.2007.06.003. PMID 18538218.
- Doherty, R.B.; Crowley, R.A.; Health and Public Policy Committee of the American College of Physicians (2013). "Principles supporting dynamic clinical care teams: an American College of Physicians position paper". Annals of Internal Medicine 159 (9): 620–626. doi:10.7326/0003-4819-159-9-201311050-00710. PMID 24042251.
- Hersh, W.R.; Weiner, M.G.; Embi, P.J.; et al. (2013). "Caveats for the use of operational electronic health record data in comparative effectiveness research". Medical Care 51 (8 Suppl 3): S30–S37. doi:10.1097/MLR.0b013e31829b1dbd. PMC PMC3748381. PMID 23774517. http://www.pubmedcentral.nih.gov/articlerender.fcgi?tool=pmcentrez&artid=PMC3748381.
- Hersh, W.R. (2009). "A stimulus to define informatics and health information technology". BMC Medical Informatics & Decision Making (9): 24. doi:10.1186/1472-6947-9-24.
- Shekelle, P.G.; Pronovost, P.J.; Wachter, R.M.; et al. (2013). "The top patient safety strategies that can be encouraged for adoption now". Annals of Internal Medicine 158 (5 Pt 2): 365–368. doi:10.7326/0003-4819-158-5-201303051-00001. PMID 23460091.
- Committee on Patient Safety and Health Information Technology; Board on Health Care Services; Institute of Medicine (2012). Health IT and Patient Safety: Building Safer Systems for Better Care. Washington, DC: National Academies Press. ISBN 9780309221122.
- Ricciardi, L.; Mostashari, F.; Murphy, J.; Daniel, J.G.; Siminerio, E.P. (2013). "A national action plan to support consumer engagement via e-health". Health Affairs 32 (2): 376–384. doi:10.1377/hlthaff.2012.1216. PMID 23381531.
- Institute for Health Technology Transformation (2012) (PDF). Population Health Management – A Roadmap for Provider-Based Automation in a New Era of Healthcare. New York, NY: Institute for Health Technology Transformation. http://ihealthtran.com/pdf/PHMReport.pdf.
- Diamond, C.C.; Mostashari, F.; Shirky, C. (2009). "A national action plan to support consumer engagement via e-health". Health Affairs 28 (2): 454–466. doi:10.1377/hlthaff.28.2.454. PMID 19276005.
- Hamburg, M.A.; Collins, F.S. (2010). "The path to personalized medicine". New England Journal of Medicine 363 (4): 301–304. doi:10.1056/NEJMp1006304. PMID 20551152.
- Fernald, G.H.; Capriotti, E.; Daneshjou, R.; Karczewski, K.J.; Altman, R.B. (2011). "Bioinformatics challenges for personalized medicine". Bioinformatics 27 (13): 1741–1748. doi:10.1093/bioinformatics/btr295. PMC PMC3117361. PMID 21596790. http://www.pubmedcentral.nih.gov/articlerender.fcgi?tool=pmcentrez&artid=PMC3117361.
- Manolio, T. (2010). "Genomewide association studies and assessment of the risk of disease". New England Journal of Medicine 363 (2): 166–176. doi:10.1056/NEJMra0905980. PMID 20647212.
This presentation is faithful to the original, with only a few minor changes to presentation. In all cases the DOI, PubMed ID, and/or ISBN were missing and were added to make the reference more useful.