Journal:The state of open-source electronic health record projects: A software anthropology study
|Full article title||The state of open-source electronic health record projects: A software anthropology study|
|Journal||JMIR Medical Informatics|
|Author(s)||Alsaffar, Mona; Yellowlees, Peter; Odor, Alberto; Hogarth, Michael|
|Author affiliation(s)||University of California-Davis|
|Primary contact||Email: mona_alsaffar [at] hotmail dot com; Phone: 1 9167348710|
|Volume and issue||5 (1)|
|Distribution license||Creative Commons Attribution 2.0|
- 1 Abstract
- 2 Introduction
- 3 Methods
- 4 Results
- 5 Discussion
- 5.1 Principal findings
- 5.2 Challenges
- 5.3 Scope and limitations
- 6 Conclusion
- 7 Conflicts of interest
- 8 Multimedia Appendix 1
- 9 References
- 10 Abbreviations
- 11 Notes
Background: Electronic health records (EHR) are a key tool in managing and storing patients’ information. Currently, there are over 50 open-source EHR systems available. Functionality and usability are important factors for determining the success of any system. These factors are often a direct reflection of the domain knowledge and developers’ motivations. However, few published studies have focused on the characteristics of free and open-source software (F/OSS) EHR systems, and none to date have discussed the motivation, knowledge background, and demographic characteristics of the developers involved in open-source EHR projects.
Objective: This study analyzed the characteristics of prevailing F/OSS EHR systems and aimed to provide an understanding of the motivation, knowledge background, and characteristics of the developers.
Methods: This study identified F/OSS EHR projects on SourceForge and other websites from May to July 2014. Projects were classified and characterized by license type, downloads, programming languages, spoken languages, project age, development status, supporting materials, top downloads by country, and whether they were "certified" EHRs. Health care F/OSS developers were also surveyed using an online survey.
Results: At the time of the assessment, we uncovered 54 open source EHR projects, but only four of them had been successfully certified under the Office of the National Coordinator for Health Information Technology (ONC Health IT) Certification Program. In the majority of cases, the open-source EHR software was downloaded by users in the United States (64.07%, 148,666/232,034), underscoring that there is a significant interest in EHR open-source applications in the United States. A survey of EHR open source developers was conducted, and a total of 103 developers responded to the online questionnaire. The majority of EHR F/OSS developers (65.3%, 66/101) are participating in F/OSS projects as part of a paid activity, and only 25.7% (26/101) of EHR F/OSS developers are, or have been, health care providers in their careers. In addition, 45% (45/99) of developers do not work in the health care field.
Conclusion: The research presented in this study highlights some challenges that may be hindering the future of health care F/OSS. A minority of developers have been health care professionals, and only 55% (54/99) work in the health care field. This undoubtedly hinders the functional design of F/OSS EHR systems from exhibiting a competitive advantage over prevailing commercial EHR systems. Open-source software seems to be a significant interest to many; however, given that only four F/OSS EHR systems are ONC-certified, this interest is unlikely to yield significant adoption of these systems in the United States. Although the Health Information Technology for Economic and Clinical Health (HITECH) Act was responsible for a substantial infusion of capital into the EHR marketplace, the lack of a corporate entity in most F/OSS EHR projects translates to a marginal capacity to market the respective F/OSS system and to navigate certification. This likely has further disadvantaged F/OSS EHR adoption in the United States.
Keywords: open source, electronic health record, SourceForge, developers, motivations
The medical field has been using open-source applications for almost 40 years. Electronic health record (EHR) systems first appeared in the early 1960s. The Computer Stored Ambulatory Record (COSTAR) system was the first F/OSS EHR system and was originally developed to be used by the Harvard Community Health Plan. Although COSTAR was implemented in a number of institutions, it did not result in broad national adoption of EHRs at the time. Only the Health Information Technology for Economic and Clinical Health (HITECH) Act of 2009 and its financial incentive program have resulted in broad adoption of EHRs in the United States. F/OSS EHR systems have been increasing in popularity over the period.
Although the HITECH incentive payments have increased adoption, EHR adoption continues to have obstacles. One of the main obstacles continues to be affordability. CDW Healthcare Physician Practice estimated the total cost of an EHR deployment at approximately USD $120,000 per physician in the first year after implementation, with annual recurring costs of USD $30,000 per physician. Along with the financial cost, there is also the non-financial cost related to time spent to bring the system live and into full functional use.
Open-source EHR may lessen financial barriers while also providing improved flexibility, given that they can be "freely" modified. Many of the prevailing EHRs do not adhere to minimal usability testing standards, requiring continuous customization to meet the needs of the organization. A KLAS study of 128 physicians on the current state of acute care EHRs found that no vendor scored high in usability. Since open-source software can be freely modified and redistributed, this could reduce the cost of continuous customization to improve usability. Open-source projects tend to also benefit from a higher degree of transparency about software anomalies (software bugs), leading to a higher degree of reliability over time. A common belief across the open source community, and often referred to as "Linus Law," states "given enough eyeballs, all bugs are shallow." Unlike organizations who are dependent on a commercial vendor’s prioritization of features and software release schedules, those implementing F/OSS would have complete control over the timing of customization and deployment, allowing them to choose what functionality is available and when it will be available to their users.
F/OSS does come with challenges as well. Although some commercial companies provide support for F/OSS EHRs, the majority of the F/OSS EHR projects do not have a support service one can purchase. This creates a major challenge in ensuring reliability, particularly when the original system has been customized by institutional programmers. Those skeptical of F/OSS EHR systems often highlight the potential dependency on volunteer developers who do not guarantee technical support. In addition, identifying a reliable source for version updates can be challenging. Many organizations also fear that open-source projects can become inactive anytime, creating an acute need for substantial in-house software development expertise. A majority of health care organizations do not typically have infrastructure to support software development, and they might not have information technology (IT) staff with expertise in managing the software development lifecycle (SDLC) for complex systems. Instead, the typical health care delivery organization’s IT staff focuses on deploying and optimizing vendor software.
Despite these disadvantages, F/OSS software has been growing in terms of the number of projects. The Eighth Annual Future of Open Source Survey found that the number of F/OSS projects doubled between 2012 and 2014.
The core success of the open source movement depends on developers who contribute their knowledge and effort for free to the community. Developers are either unpaid volunteers, hobbyists, or employees who are paid to write code. A study of mainstream F/OSS projects categorized developers’ contribution into eight different roles: project leader, core member, active developers, peripheral developer, bug fixer, bug reporter, reader, and passive user. As reflected in this categorization, there are a number of different roles for contributors and a significant amount of resources required to support a high-quality project. In large part, the developer community and their motivations are a key determinant of success or failure of an open-source project. Exploring these motivations is an important aspect of understanding a key success factor for F/OSS EHR systems. The motivation-affecting factors for open source developers can be categorized as internal (cognitive) and external (social). Internal factors are comprised of motivation, altruism, and community identification. External motivation factors include future rewards (e.g., peer recognition), self-marketing, human capital, contribution as part of employment (i.e., being paid to contribute), and revenue from related products and services. Generally, anything related to the joy of coding is considered intrinsic motivation, whereas extrinsic motivation is associated with receiving some benefit for the contribution. These factors have been explored in mainstream F/OSS projects but have not yet been characterized in F/OSS health care projects.
The objectives of this study were to canvass the current state of open-source EHR systems and to characterize the motivations, knowledge, and demographics of the developers.
To find EHR F/OSS projects, we used SourceForge, a widely used open source project repository, and Google, using the search terms "electronic patient record," "electronic health record," "electronic medical records," and "clinical information system." The search revealed hundreds of EHR F/OSS projects, but only 54 of them were EHRs according to our study inclusion criteria. The following are two fundamental inclusion criteria used in the study:
- The software had to be defined as an EHR, such that the project had to adhere to the functional definition of EHR. The HealthIT.gov website defines an EHR as a "digital chart" containing at least the medical and treatment history of the patients; and
- The software had to use an open-source license. Open-source software is defined as software without license restrictions on its redistribution, and the software can be freely modified.
The study was conducted for a three-month period starting May 2014. To understand the characteristics of the various EHR projects, we looked at license type, downloads, programming languages, spoken languages, project age, development status, supporting materials, top downloads by country, and whether they were "certified" EHRs.
Many SourceForge applications are defined by their license type on the application homepage. In this study, the licenses were classified into permissive, restrictive, or highly restrictive. The highly restrictive licenses, such as a general public license (GPL), allow free modification but request that any modification should be contributed back to the community under the same license. Highly restrictive licenses are used more in applications geared toward the end user (e.g., games). License restrictions tend to affect who contributes and accessibility of the source code.
We made an assumption that download frequency reflects the popularity of the software. This assumption is a commonly held belief in this research domain. We looked at the download number in the last 12-month period on SourceForge. Around 47 projects (87%, 47/54) in this study have information on downloads, a proxy for use of the software.
Development status shows the readiness of software for day to-day use. This study utilized the SourceForge classification for software readiness. In this study, six software stages were used: planning, pre-alpha, alpha, beta, production, stable, and mature. The software’s status can influence a project’s success and affects the interest of the users and developers.
Project age represents the number of years since the project development started. The project’s age, in addition to other factors, is positively related to its ability to attract more users and/or resources, which affect the project’s future sustainability.
Programming language for each software system was examined and classified according to whether one or multiple were used. One open source study suggests that using one common programming language affects the success of the software project.
The projects were classified according to their spoken languages. One study proves that open source software popularity is related to the number of language versions available.
The setup of the system is not always obvious, and in some cases it requires IT administration skills. The top 10 downloaded projects were analyzed, as these materials make the installation and usage of the system easier. The supporting materials included user guides, installation guides, and version demonstrations.
Top downloads according to countries
SourceForge provides important information about the software and gives the highest number of downloads for each EHR software system. The top 10 most downloaded projects were identified, in addition to the country that made the greatest number of downloads, to see if the adoption rate affects the number of downloads.
Certified open source electronic health record
Using a certified EHR is a requirement for payments through the HITECH Act’s EHR-incentive program. Certification of an EHR under the ONC program is an important success factor in the United States. The certified open-source EHR products and their specifications from the Centers for Medicare and Medicaid Services (CMS) website were examined (Table 1).
Survey data collection
A survey was conducted in 2014 using the commercial survey tool SurveyGizmo (Boulder, CO). The survey consisted of 20 questions and took approximately five minutes to complete (Multimedia Appendix 1). The target audience included anyone who self-identified as a developer of health care F/OSS. Our questionnaire was modeled after a similar survey developed to compare proprietary and open-source software in 2003.
An announcement of the survey was published on 10 websites that focused on health care open source news and targeted health care developers. The announcement contained a brief summary of the main goal of the survey along with the author's names and their affiliations. About one week after this first announcement, the survey was distributed to 54 open-source project developers’ mailing lists obtained from SourceForge. We reached out to project email addresses and asked them to distribute our survey to their mailing lists. In addition, we sent a personal survey invitation to specific developers who mentioned working in F/OSS health care projects in their LinkedIn profile. The survey was posted for five weeks, and a total of 103 responses were collected.
The study revealed several key observations. At the time of the study, there were 54 open source EHR projects, but only four had been successfully certified under the Office of the National Coordinator for Health Information Technology (ONC Health IT) Certification Program. Nearly half of the projects (57%, 31/54) used a restrictive license type, and approximately 57% (30/54) used GPL. The data revealed that 52% (28/54) of the projects were in production/stable status, only 2 (4%, 2/54) were in mature status, while 1 (2%, 1/54) project was inactive (Table 2). There were 44 active projects at varying stages of development, while 10 had unspecified status.
As one might expect, many open-source projects (46%, 25/54) used one programming language. However, a large percentage (36%, 19/54) used multiple programming languages. Approximately 18% (10/54) of the projects did not indicate the use of a specific programming language. The analysis also showed that the number of downloads for the projects that were written using multiple programming languages (n=147,914) were higher than the projects using one programming language (n=115,299). Among those projects indicating a programming language, the "PHP Hypertext Preprocessor," commonly known as PHP, was the leader, and it was used in 31% (17/54) of the projects.
The OpenEMR project had the highest number of downloads (63,418 in a 12-month period) (Table 3). The data shows that the United States accounts for the majority of the downloads and constitutes 64.07% (148,666/232,034) of the total downloads of open-source EHR projects on SourceForge. In total, 19% (10/54) of open-source EHR systems have installation and user guides along with demonstration versions. The mean project age is seven years, with a range of two to 14 years.
In terms of project (spoken) language, a large number of projects were written in English (46%, 25/54), but surprisingly 28% (15/54) were being developed in one or more languages besides English. It was found that 11% (6/54) of the projects did not specify a language (Table 4). Data shows that the number of downloads for projects with multiple spoken languages was very high (n=187,933); with a download rate three times the one-language projects (n=48,402). These numbers indicate a global interest in F/OSS EHR development and prove that the number of language translations is positively related to project success and popularity.
A total of 103 developer survey responses were successfully collected showing the developer’s characteristics, background, and their reasons to contribute to health care open-source projects. Survey respondents were primarily male (94%, 94/99) with an age range of 25 to 55 years old. The majority of developers were American (37%, 36/98) and 42% (41/98) lived in the United States. Employees made up 58% (58/100) of the sample, and self-employed developers constituted 33.0% (33/100). Approximately 58% (58/99) of developers were married and 18.0% (18/100) had children older than age six. As F/OSS developers tend to be highly educated, many had graduate level (43.0%, 43/100), professional level (18.0%, 18/100), or undergraduate level (36.0%, 36/100) education. Only 3.0% (3/100) of F/OSS developers reported not having formal education beyond high school.
As has been found in mainstream F/OSS projects, the majority of health care F/OSS developers participated in the projects as part of a paid activity. Nearly 34.7% (35/101) of the contributors received direct payment for developing F/OSS. Another 30.7% (31/101) received direct payment for either managing or supporting F/OSS while 34% (32/94) received no payment at all for contributing to the projects. Most of the developers contributed by writing code during their off-work hours (41.6%, 42/101) and (39.6%, 40/101) during their work hours. Almost half of the respondents reported employer awareness of their F/OSS work (42%, 42/99). A number of respondents (32%, 32/99) worked on F/OSS as part of their employment. Some employers (17%, 17/99) were unaware of the developers work on F/OSS, but very few (3%, 3/99) did not want them to contribute to F/OSS development.
Contributors’ efforts have been measured by the number of hours spent on a project per week. Respondents’ answers show an average of 9.8 hours per week spent on F/OSS projects. Health care open source developers were asked how many projects they had contributed to. Nearly 49.0% (50/102) worked on one project, while 47% (48/94) had worked on two to five open-source projects. The average number of projects was 1.6, with a maximum number of five projects.
A significant number of respondents (47.0%, 47/100) reported 75% to 100% of their code was included in a F/OSS project. Around one third (43.0%, 43/100) reported less than 25% of their code was included in the final project. The majority of contributors (50%, 48/96) wrote between 500 and 5000 code lines, while only 30% (29/96) wrote more than 10,000 lines. The average lines of code were 1776, with a maximum of 5000 lines.
Respondents were asked to answer several questions regarding their F/OSS project to analyze their opinions, motivations, and habits toward F/OSS projects. Interestingly, developers had different motivations to participate in F/OSS. The top reasons for contributing to a health care open-source project were based on enjoyment-related intrinsic motivation; the project was "important and visible" (47.5%, 48/101) or "technically interesting" (47.5%, 48/101) for them. Approximately 20.8% (21/101) indicated community-based related intrinsic motivation as their reason to contribute and stated that they knew people working on the project.
The survey results confirmed that many developers start developing F/OSS to give back to the community. Around 40% (39/97) considered it important or very important to give back to the community and 38% (37/96) considered the interaction with like-minded programmers to be important. A significant number of respondents were motivated to promote the mode of development and the ideal of freely modifiable software (47%, 47/99), while the remaining developers were motivated to provide alternatives to proprietary software (53%, 50/93). Some developers (27%, 25/97) began developing F/OSS software by modifying it to fit with their requirements or to fix the bugs in their existing software (32 %, 32/97), and 26% (25/96) of them were interested in learning how the program worked.
Open-source projects continue being started at a rapid pace, but sustainability of the projects appears to be a challenge. A study in 2005 found 45 F/OSS EHR applications compared to the 54 found in this study. This represents a 16.6% increase in the past nine years. However, most of the projects are different, suggesting sustainability is a significant challenge. Functional EHR systems are complex with many required subsystems and modules, requiring robust development and change management processes in order to achieve high-quality "production-grade" software for the mission-critical environment of a hospital or clinic. Certification is an added challenge in some markets. Despite these challenges, F/OSS EHR systems have been viewed as excellent options for community health clinics, small practices, and hospitals that are under-capitalized in the United States and overseas.
In terms of open-source licenses, the majority of F/OSS projects use GPL, which allows free modification but requires that any modification be contributed back to that open-source project under the same license. This creates a "poison pill," making it difficult for commercial entities to integrate GPL open-source components into a module that also has proprietary software. This can be viewed as a "restriction" with regards to open-source licensing. Restrictiveness of the license also affects a number of contributors and accessibility of the source code.
With regards to developers, this study found the average F/OSS contributor is well-educated, young, and male; the F/OSS community is male dominant, as women contributors are only 2%. The reasons for gender inequities remain unclear but may involve women facing hybrid discrimination from a F/OSS community. Our research shows marital status and having kids older than six years old increases the probability a person will volunteer.
One of the most common aspects of F/OSS is that they are based on voluntary efforts of developers. The goal of this research was to understand what motivates these developers to contribute to a health care F/OSS project. Learning new skills and becoming a better programmer is one of the motivations this research found (35%, 34/96), which is similar to previous findings about mainstream F/OSS developers (36.5%). Learning and acquiring new skills appears to be another important motivation for many developers.
Intrinsic motivations, such as altruism, are high among health care developers; 47.5% (48/101) of developers worked on the programs for this reason. Only 16% of developers in other research papers stated that altruism was their prime motivator. Contrary to our research findings, some previous research papers found that only a minority were motivated by extrinsic motivation, which involved payments for their participation in F/OSS projects.
One of the unique findings in this research is that 74.2% (75/101) of health care F/OSS developers are not health care practitioners, and 45% (45/99) do not work in the health care field. Being a developer outside of the health care field can be a core problem for the development of usable clinical software with a high degree of functionality. This may serve to explain why open-source EHRs have limited functionality today.
Although the open-source projects solve licensing cost problems, there is a need for maintenance and implementation costs, which can be a barrier for organizations without health IT expertise. There are many open-source EHR options on SourceForge, but not all of them can fit in the clinical workflow in the United States or in certain hospitals.
The availability of motivated developers and the need to continuously improve EHR systems will likely mean F/OSS EHR will continue to be part of the health care software landscape despite the many challenges these projects face today. Characterizing the challenges and benefits of adopting open-source EHR will be important in understanding the value of these systems.
This study shows a potential weakness for F/OSS EHR projects stemming from developer background and their ability to understand some of the nuances in health care workflows. A large fraction of developers (74.2%, 75/101) were not health care practitioners, and few developers (54%, 54/99) had worked in the health care field. A potential solution to this issue would be to have programs that give F/OSS EHR developers direct access to providers and care venues.
Many hospitals would benefit greatly from integrated software, rather than a disparate group of systems. However, the F/OSS option can be difficult to interface with commercial systems and may require personnel with multiple skill sets.
Privacy and security
Maintaining patient privacy through robust security is a critical aspect of any EHR system. A study of F/OSS health care systems found the information is often not safeguarded with consent or privacy policies and offers limited protection against unauthorized access or release of information. However, we did find one open-source EHR system (Tolven eCHR), which supported encryption of health care data at the row level in a relational database. This would make data unreadable even to the database administrator of the system. This is a high degree of security and not typical among EHR systems at the time.
Lack of financial and professional expertise
Low acquisition, installation, and ownership costs make the F/OSS system an excellent option for organizations with limited capital. However, in general, the health care environments that could benefit the most from F/OSS EHR systems tend to have low information technology capital budgets and very limited access to health informatics professionals. These challenges may prevent the successful implementation of any health IT technology, including F/OSS EHR. Our study shows that developing countries adopted F/OSS EHR systems, but the major adoptions were in North America after the HITECH Act. This suggests that even F/OSS EHR system implementations require substantial financial and workforce resource capacity to succeed.
Although F/OSS software does not have licensing costs, effective implementation still requires skilled staff, time for installation, and time for learning the software. F/OSS EHR implementation cost can be as high as the proprietary software because of the add-ons, consultation costs, and need for assistance. An important aspect of the total cost of ownership of an EHR system is usability. Poor usability can directly impact the productivity of expensive health care providers and support personnel. The ability to freely modify F/OSS EHR systems and optimize them for local use likely translates into lower overall cost.
Skilled information technology personnel
Adopting F/OSS EHR requires a large number of IT employees with specific programming skills who understand the program well. A hospital that chooses to use F/OSS EHR will need to hire developer IT staff and contract with extra IT vendors or consultant support.
Limited functionality of clinical decision support
At an operational level, F/OSS software also presented reduced functions in decision support and knowledge management. Clinical reasoning, guidelines and protocols, quality assurance, and integrated care were rather limited or nonexistent in most applications. Developers lack medical knowledge as the majority of them are not health care practitioners, which may affect the efficacy of clinical decision support functionality.
Lack of liability and accountability
One of the major shortcomings is the lack of liability and accountability in F/OSS. Our study did not address this issue. Few studies of F/OSS in health care have addressed the risks of using F/OSS. F/OSS EHR projects come without warranties regarding the development, release date, or fulfillment of functionality. F/OSS EHR systems do not have a commercial entity providing support and tend to rely on a volunteer community. This can be a significant risk for a software system that supports the core business of a hospital.
Scope and limitations
Data on F/OSS EHR projects were primarily collected from SourceForge. Therefore, this study is a snapshot in time, with projects being added and deleted before or after data collection. This amount of "churn" reflects a dynamic software category where our findings may not apply in the future.
Not all F/OSS EHR projects were listed on SourceForge; some projects are managed through independent websites. Therefore, some of the projects on SourceForge were outdated, may have contained inaccurate information, and were a small representation of F/OSS EHR projects as a whole. SourceForge was the main source for most of the data because it tends to be one of the preferred project management platforms for the open source movement and has numerous projects and registered developers.
Survey questions were mostly closed-ended which restricted respondents’ answer options. However, we did accommodate all possible answers by providing an additional "other" option for some questions. There is a possibility that respondents misunderstood questions as there was no usability testing done on the survey prior to using it in the general F/OSS EHR community; however, we did not receive any respondent requests to clarify any questions. Furthermore, the survey respondents do not reflect the entire population of open source developers, but we believe it is a reasonable representative of the target population.
This study highlights a number of important aspects of F/OSS EHR. Open-source software systems seem to be important to some health care organizations; however, only four F/OSS EHR systems are ONC-certified in the United States, which creates a barrier to broader use. Health care open-source software also currently lacks directed corporate or governmental support for sustainability and growth of these software programs. We hope this research underlines the challenges that hinder the future of F/OSS and provides avenues for future research to study and improve adoption of F/OSS systems in the United States.
Conflicts of interest
Multimedia Appendix 1
- McDonald, C.J.; Schadow, G.; Barnes, M. et al. (2003). "Open source software in medical informatics -- Why, how and what". International Journal of Medical Informatics 69 (2–3): 175–84. doi:10.1016/S1386-5056(02)00104-1. PMID 12810121.
- Barnett, G.O.; Justice, N.S.; Somand, M.E. et al. (1978). "COSTAR — A computer-based medical information system for ambulatory care". Proceedings of the Annual Symposium on Computer Applications in Medical Care 1978: 486–487. doi:10.1109/SCAMC.1978.679955. PMC PMC2231744. http://www.pubmedcentral.nih.gov/articlerender.fcgi?tool=pmcentrez&artid=PMC2231744.
- Henry, J.W.; Pylypchuk, Y.; Searcy, T. et al. (May 2016). "Adoption of Electronic Health Record Systems among U.S. Non-Federal Acute Care Hospitals: 2008-2015". Health IT Dashboard. Office of the National Coordinator for Health Information Technology. https://dashboard.healthit.gov/evaluations/data-briefs/non-federal-acute-care-hospital-ehr-adoption-2008-2015.php. Retrieved 21 September 2016.
- Ye, Y.; Kishida, K. (2003). "Toward an understanding of the motivation open source software developers". Proceedings of the 25th International Conference on Software Engineering 2003: 419-29. ISBN 076951877X.
- Gans, D.; Kralewski, J.; Hammons, T. et al. (2005). "Medical groups' adoption of electronic health records and information systems". Health Affairs 24 (5): 1323-33. doi:10.1377/hlthaff.24.5.1323. PMID 16162580.
- Ajami, S.; Bagheri-Tadi, T. (2013). "Barriers for adopting electronic health records (EHRs) by physicians". Acta Informatica Medica 21 (2): 129–34. doi:10.5455/aim.2013.21.129-134. PMC PMC3766548. PMID 24058254. http://www.pubmedcentral.nih.gov/articlerender.fcgi?tool=pmcentrez&artid=PMC3766548.
- Fleming, N.S.; Culler, S.D.; McCorkle, R. et al. (2011). "The financial and nonfinancial costs of implementing electronic health records in primary care practices". Health Affairs 30 (3): 481-9. doi:10.1377/hlthaff.2010.0768. PMID 21383367.
- Aminpour, F.; Sadoughi, F.; Ahamdi, M. (2014). "Utilization of open source electronic health record around the world: A systematic review". Journal of Research in Medical Sciences 19 (1): 57-64. PMC PMC3963324. PMID 24672566. http://www.pubmedcentral.nih.gov/articlerender.fcgi?tool=pmcentrez&artid=PMC3963324.
- Ratwani, R.M.; Benda, N.C.; Hettinger, A.Z. et al. (2015). "Electronic health record vendor adherence to usability certification requirements and testing standards". JAMA 314 (10): 1070-1. doi:10.1001/jama.2015.8372. PMID 26348757.
- Briggs, B.; Carter-Templeton, H. (2014). "Electronic health record customization: A quality improvement project". Online Journal of Nursing Informatics 18 (3). http://www.himss.org/electronic-health-record-customization-quality-improvement-project.
- Tate, C. (June 2015). "AMDIS 2015" (PDF). KLAS Enterprises, LLC. http://amdis.org/wp-content/uploads/2015/06/Klas-Review_Tate1.pdf. Retrieved 21 September 2016.
- Yellowlees, P.M.; Marks, S.L.; Hogarth, M. et al. (2008). "Standards-based, open-source electronic health record systems: A desirable future for the U.S. health industry". Telemedicine Journal and E-Health 14 (3): 284-8. doi:10.1089/tmj.2007.0052. PMID 18570554.
- Raymond, E. (1999). "The cathedral and the bazaar". Knowledge, Technology & Policy 12 (3): 23–49. doi:10.1007/s12130-999-1026-0.
- Kobayashi, S. (2012). "Open Source Software Development on Medical Domain". In Kalloniatis, C.. Modern Information Systems. InTech. pp. 1–16. doi:10.5772/38117. ISBN 9789535106470.
- Maglogiannis, I. (2012). "Towards the adoption of open source and open access electronic health record systems". Journal of Healthcare Engineering 3 (1): 141-161. doi:10.1260/2040-22188.8.131.52.
- Chełkowski, T.; Gloor, P.; Jemielniak, D. (2016). "Inequalities in open source software development: analysis of contributor's commits in Apache Software Foundation projects". PLOS One 11 (4): e0152976. doi:10.1371/journal.pone.0152976. PMC PMC4838325. PMID 27096157. http://www.pubmedcentral.nih.gov/articlerender.fcgi?tool=pmcentrez&artid=PMC4838325.
- Goulde, M.; Brown, E.; Rymer, J.; Hartzband, D. (March 2006). "Open Source Software: A Primer for Health Care Leaders". iHealth Reports. Forrester Consulting. pp. 31. http://www.chcf.org/publications/2006/03/open-source-software-a-primer-for-health-care-leaders. Retrieved 07 February 2017.
- "The Future of Open Source 2012 Survey Results". North Bridge Venture Partners. 21 May 2012. http://www.northbridge.com/2012-open-source-survey. Retrieved 21 September 2016.
- "The Future of Open Source 2014 Survey Results". North Bridge Venture Partners. 2014. http://www.northbridge.com/2014-future-open-source-survey-results-0. Retrieved 21 September 2016.
- Godfrey, M.W.; Tu, Q. (2000). "Evolution in open source software: A case study". Proceedings of the International Conference on Software Maintenance 2000: 131. doi:10.1109/ICSM.2000.883030. ISBN 0769507530.
- Hars, A.; Ou, S. (2002). "Working for free? Motivations for participating in open-source projects". International Journal of Electronic Commerce 6 (3): 25–39. doi:10.1109/HICSS.2001.927045.
- Sen, R.; Subramaniam, C.; Nelson, M. (2008). "Determinants of the choice of open source software license". Journal of Management Information Systems 25 (3): 207–40. doi:10.2753/MIS0742-1222250306.
- Lerner, J.; Tirole, J. (2002). "The scope of open source licensing". Journal of Law, Economics and Organization 21 (1): 20–56. doi:10.3386/w9363.
- Janamanchi, B.; Katsamakas, E.; Raghupathi, W. et al. (2009). "The state and profile of open source software projects in health and medical informatics". International Journal of Medical Informatics 78 (7): 457-72. doi:10.1016/j.ijmedinf.2009.02.006. PMID 19321384.
- Chengalur-Smith, I.; Sidorova, A.; Daniel, S. (2010). "Sustainability of free/libre open source projects: A longitudinal study". Journal of the Association for Information Systems 11 (11): 5. http://aisel.aisnet.org/jais/vol11/iss11/5/.
- Crowston, K.; Scozzi, B. (2002). "Open source software projects as virtual organisations: Competency rallying for software development". IEE Proceedings - Software 149 (1): 3–17. doi:10.1049/ip-sen:20020197.
- Midha, V.; Palvia, P. (2012). "Factors affecting the success of open source software". Journal of Systems and Software 85 (4): 895-905. doi:10.1016/j.jss.2011.11.010.
- Arora, S.; Waterman, A.; David, P.. "Free/Libre/Open Source Software Group - Survey". Stanford. http://web.stanford.edu/group/floss-us/questionnaire/FLOSS-US-Questionnaire.html. Retrieved 21 September 2016.
- Hogarth, M.A.; Turner, S. (2005). "A study of clinically related open source software projects". AMIA Annual Symposium Proceedings 2005: 330–4. PMC PMC1560618. PMID 16779056. http://www.pubmedcentral.nih.gov/articlerender.fcgi?tool=pmcentrez&artid=PMC1560618.
- Karopka, T.; Schmuhl, H.; Demski, H. (2014). "Free/Libre open source software in health care: A review". Healthcare Informatics Research 20 (1): 11–22. doi:10.4258/hir.2014.20.1.11. PMC PMC3950260. PMID 24627814. http://www.pubmedcentral.nih.gov/articlerender.fcgi?tool=pmcentrez&artid=PMC3950260.
- Mahmod, M.; Yusof, S.; Dahalin, Z. (2010). "Women contributions to open source software innovation: A social constructivist perspective". 2010 International Symposium on Information Technology 2010: 1433-1438. doi:10.1109/ITSIM.2010.5561496.
- Selbee, L.K.; Reed, P.B. (2000). "Patterns of Volunteering Over the Life Cycle" (PDF). Statistics Canada. http://www3.carleton.ca/casr/Patterns%20of%20Vol.pdf. Retrieved 21 September 2016.
- Hayghe, H.V. (1991). "Volunteers in the U.S.: Who donates the time?". Monthly Labor Review 114 (2): 17–23. PMID 10110196.
- Lakhani, K.R.; Wolf, R. (2005). "Chapter 1: Why Hackers Do What They Do: Understanding Motivation and Effort in Free/Open Source Software Projects". In Feller, J.; Fitzgerald, B.; Hissam, S.; Lakhani, K.R.. Perspectives on Free and Open Source Software. MIT Press. pp. 3–22. ISBN 9780262062466.
- Luthiger, B.; Jungwirth, C. (2007). "Pervasive fun". First Monday 12 (1). doi:10.5210/fm.v12i1.1422.
- Schmuhl, H.; Heinze, O.; Bergh, B. (2013). "Use of open source software in health care delivery - Results of a qualitative field study". Yearbook of Medical Informatics 8: 107–113. PMID 23974556.
- Flores Zuniga, A.E.; Win, K.T.; Susilo, W. (2010). "Functionalities of free and open electronic health record systems". International Journal of Technology Assessment in Health Care 26 (4): 382–9. doi:10.1017/S0266462310001121. PMID 20974022.
COSTAR: Computer Stored Ambulatory Record
EHR: electronic health record
F/OSS: free and open source software
GPL: general public license
HITECH: Health Information Technology for Economic and Clinical Health
IT: information technology
ONC: Office of the National Coordinator
This presentation is faithful to the original, with only a few minor changes to presentation. In several cases the PubMed ID and DOI was missing and were added to make the references more useful.
Per the distribution agreement, the following copyright information is also being added:
©Mona Alsaffar, Peter Yellowlees, Alberto Odor, Michael Hogarth. Originally published in JMIR Medical Informatics (http://medinform.jmir.org), 24.02.2017.