Difference between revisions of "Journal:Determining the hospital information system (HIS) success rate: Development of a new instrument and case study"

From LIMSWiki
Jump to: navigation, search
(Saving and adding more.)
(Saving and adding more.)
(5 intermediate revisions by the same user not shown)
Line 41: Line 41:
  
 
In 2014, the Iranian Ministry of Health and Medical Education mandated that public hospitals transfer patients’ medical records to SEPAS (Iran’s [[electronic health record]] [EHR] project). This led to a rapid growth in the implementation of a HIS in hospitals across the country. Following the implementation of EHRs and HISs in Saudi Arabia<ref name="AlharthiPhys14">{{cite journal |title=Physician satisfaction with electronic medical records in a major Saudi Government hospital |journal=Journal of Taibah University Medical Sciences |author=Alharthi, H.; Youssef, A.; Radwan, S. et al. |volume=9 |issue=3 |pages=213–18 |year=2014 |doi=10.1016/j.jtumed.2014.01.004}}</ref>, Korea<ref name="IqbalTheRelat13" />, and Taiwan<ref name="WangAttitudes15">{{cite journal |title=Attitudes toward inter-hospital electronic patient record exchange: Discrepancies among physicians, medical record staff, and patients |journal=BMC Health Services Research |author=Wang, J.Y.; Ho, H.Y.; Chen, J.D. et al. |volume=15 |pages=264 |year=2015 |doi=10.1186/s12913-015-0896-y |pmid=26162748 |pmc=PMC4499194}}</ref><ref name="LuFactors12">{{cite journal |title=Factors determining nurse acceptance of hospital information systems |journal=Computers, Informatics, Nursing |author=Lu, C.H.; Hsiao, J.L.; Chen, R.F. |volume=30 |issue=5 |pages=257–64 |year=2012 |doi=10.1097/NCN.0b013e318224b4cf |pmid= 22228251}}</ref>, the success of these systems was evaluated. In the case of Iran, a developing country, to the best of our knowledge there have been very few studies which have evaluated the success of a HIS. The studies by Tavakoli ''et al.''<ref name="TavakoliElectro13">{{cite journal |title=Electronic medical record in central polyclinic of isfahan oil industry: a case study based on technology acceptance model |journal=Acta Informatica Medica |author=Tavakoli, N.; Jahanbakhsh, M.; Shahin, A. et al. |volume=21 |issue=1 |pages=23–5 |year=2013 |doi=10.5455/AIM.2012.21.23-25 |pmid=23572857 |pmc=PMC3610586}}</ref>, Abdekhoda ''et al.''<ref name="AbdekhodaTheEff15">{{cite journal |title=The effects of organizational contextual factors on physicians' attitude toward adoption of Electronic Medical Records |journal=Journal of Biomedical Informatics |author=Abdekhoda, M.; Ahmadi, M.; Gohari, M. et al. |volume=53 |pages=174–9 |year=2015 |doi=10.1016/j.jbi.2014.10.008 |pmid=25445481}}</ref>, and Saghaeiannejad-Isfahani ''et al.''<ref name="Saghaeiannejad-IsfahaniASurv14">{{cite journal |title=A Survey on the Users' Satisfaction with the Hospital Information Systems (HISs) based on DeLone and McLean's Model in the Medical-Teaching Hospitals in Isfahan City |journal=Acta Informatica Medica |author=Saghaeiannejad-Isfahani, S.; Jahanbakhsh, M.; Habibi, M. et al. |volume=22 |issue=3 |pages=179-82 |year=2014 |doi=10.5455/aim.2014.22.179-182 |pmid=25132711 |pmc=PMC4130683}}</ref> offer some examples to this end. Nevertheless, these studies did not collectively provide conclusive evidence on the success of HISs. As such, there is a need for additional studies to gather further evidence on the effectiveness and efficiency of the HIS in Iran.
 
In 2014, the Iranian Ministry of Health and Medical Education mandated that public hospitals transfer patients’ medical records to SEPAS (Iran’s [[electronic health record]] [EHR] project). This led to a rapid growth in the implementation of a HIS in hospitals across the country. Following the implementation of EHRs and HISs in Saudi Arabia<ref name="AlharthiPhys14">{{cite journal |title=Physician satisfaction with electronic medical records in a major Saudi Government hospital |journal=Journal of Taibah University Medical Sciences |author=Alharthi, H.; Youssef, A.; Radwan, S. et al. |volume=9 |issue=3 |pages=213–18 |year=2014 |doi=10.1016/j.jtumed.2014.01.004}}</ref>, Korea<ref name="IqbalTheRelat13" />, and Taiwan<ref name="WangAttitudes15">{{cite journal |title=Attitudes toward inter-hospital electronic patient record exchange: Discrepancies among physicians, medical record staff, and patients |journal=BMC Health Services Research |author=Wang, J.Y.; Ho, H.Y.; Chen, J.D. et al. |volume=15 |pages=264 |year=2015 |doi=10.1186/s12913-015-0896-y |pmid=26162748 |pmc=PMC4499194}}</ref><ref name="LuFactors12">{{cite journal |title=Factors determining nurse acceptance of hospital information systems |journal=Computers, Informatics, Nursing |author=Lu, C.H.; Hsiao, J.L.; Chen, R.F. |volume=30 |issue=5 |pages=257–64 |year=2012 |doi=10.1097/NCN.0b013e318224b4cf |pmid= 22228251}}</ref>, the success of these systems was evaluated. In the case of Iran, a developing country, to the best of our knowledge there have been very few studies which have evaluated the success of a HIS. The studies by Tavakoli ''et al.''<ref name="TavakoliElectro13">{{cite journal |title=Electronic medical record in central polyclinic of isfahan oil industry: a case study based on technology acceptance model |journal=Acta Informatica Medica |author=Tavakoli, N.; Jahanbakhsh, M.; Shahin, A. et al. |volume=21 |issue=1 |pages=23–5 |year=2013 |doi=10.5455/AIM.2012.21.23-25 |pmid=23572857 |pmc=PMC3610586}}</ref>, Abdekhoda ''et al.''<ref name="AbdekhodaTheEff15">{{cite journal |title=The effects of organizational contextual factors on physicians' attitude toward adoption of Electronic Medical Records |journal=Journal of Biomedical Informatics |author=Abdekhoda, M.; Ahmadi, M.; Gohari, M. et al. |volume=53 |pages=174–9 |year=2015 |doi=10.1016/j.jbi.2014.10.008 |pmid=25445481}}</ref>, and Saghaeiannejad-Isfahani ''et al.''<ref name="Saghaeiannejad-IsfahaniASurv14">{{cite journal |title=A Survey on the Users' Satisfaction with the Hospital Information Systems (HISs) based on DeLone and McLean's Model in the Medical-Teaching Hospitals in Isfahan City |journal=Acta Informatica Medica |author=Saghaeiannejad-Isfahani, S.; Jahanbakhsh, M.; Habibi, M. et al. |volume=22 |issue=3 |pages=179-82 |year=2014 |doi=10.5455/aim.2014.22.179-182 |pmid=25132711 |pmc=PMC4130683}}</ref> offer some examples to this end. Nevertheless, these studies did not collectively provide conclusive evidence on the success of HISs. As such, there is a need for additional studies to gather further evidence on the effectiveness and efficiency of the HIS in Iran.
 +
 +
In the present study, we report on the development of an instrument to evaluate the rate of HIS success based on ISSM, which can be used to compare HIS success rates in different hospitals and clinical settings. Furthermore, in a case study, we investigate the success rate of HIS based on users’ viewpoints in a teaching hospital in Iran.
 +
 +
==Methodology==
 +
This was a descriptive cross-sectional study.
 +
 +
===Instrument===
 +
A self-administrated structured questionnaire was developed based on ISSM to collect data. The questionnaire comprised of seven dimensions, including system quality, information quality, service quality, system use, usefulness, satisfaction, and net benefits. Each dimension encompasses several evaluation measures. The questions were designed based on the ISSM evaluation measures and dimensions.
 +
 +
Verification of the content validity was carried out by an expert panel. The panel consisted of nine medical informatics experts. Expert validity was measured using a content validity index (CVI) and content validity ratio (CVR). The internal consistency of each dimension was measured by Cronbach’s alpha. Pearson’s correlation was calculated to evaluate the significance of associations between dimensions.
 +
 +
The final draft of the instrument consisted of two sections. The first section includes users’ characteristics such as age, gender, education level, working hours per month, ICDL certification, work experience, computer experience, and HIS experience (n=8 questions). The second section includes 60 questions on a five-point Likert scale ranging from 1 (completely disagree) to 5 (completely agree), which covers 16 evaluation measures in seven dimensions (Table 1).
 +
 +
{|
 +
| STYLE="vertical-align:top;"|
 +
{| class="wikitable" border="1" cellpadding="5" cellspacing="0" width="80%"
 +
|-
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;" colspan="3"|'''Table 1.''' Dimensions and evaluation measures
 +
|-
 +
  ! style="padding-left:10px; padding-right:10px;"|Dimensions
 +
  ! style="padding-left:10px; padding-right:10px;"|Evaluation Measure<br />(number of questions)
 +
  ! style="padding-left:10px; padding-right:10px;"|Definition
 +
|-
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;" rowspan="4"|System quality
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Adaptability (n=1)
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|“Data from different sources can be consolidated or compared without inconsistencies.”<ref name="GoodhueTask95">{{cite journal |title=Task-Technology Fit and Individual Performance |journal=MIS Quarterly |author=Goodhue, D.L.; Thompson, R.L. |volume=19 |issue=2 |pages=213–36 |year=1995 |doi=10.2307/249689}}</ref>
 +
|-
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Reliability (n=1)
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|“The dependability and consistency of access and uptime of systems”<ref name="GoodhueTask95" /><ref name="Garcia-SmithDevelop13">{{cite journal |title=Development and initial evaluation of the Clinical Information Systems Success Model (CISSM) |journal=International Journal of Medical Informatics |author=Garcia-Smith, D.; Effken, J.A. |volume=82 |issue=6 |pages=539–52 |year=2013 |doi=10.1016/j.ijmedinf.2013.01.011 |pmid=23497819}}</ref>
 +
|-
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Availability (n=6)
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Access to the computer resource and infrastructure where and when it is needed
 +
|-
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Usability (n=6)
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|“The capability in human functional terms to be used easily and effectively by the specified range of users, given specified training and user support, to fulfill the specified range of tasks, within the specified range of environmental scenarios”<ref name="ShackelHuman91">{{cite book |chapter=Usability—Context, framework, definition, design and evaluation |title=Human Factors for Informatics Usability |author=Shackel, B. |publisher=Cambridge University Press |pages=21–37 |year=1991 |isbn=0521365708}}</ref>
 +
|-
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;" rowspan="5"|Information quality
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Security (n=3)
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|“The safeguarding of data from misappropriation or unauthorized alteration or loss”<ref name="BaileyDevelop83">{{cite journal |title=Development of a Tool for Measuring and Analyzing Computer User Satisfaction |journal=Management Science |author=Bailey, J.E.; Pearson, S.W. |volume=29 |issue=5 |pages=519–637 |year=1983 |doi=10.1287/mnsc.29.5.530}}</ref>
 +
|-
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Ease of understanding (n=3)
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|“Ease of determining what a data element on a report or file means, or what is excluded or included in calculating it”<ref name="GoodhueTask95" />
 +
|-
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Completeness (n=2)
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|“The comprehensive of the output information content”<ref name="BaileyDevelop83" />
 +
|-
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Personalization (n=1)
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Ability to personalize HIS for users
 +
|-
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Relevance (n=1)
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|“The degree of congruence between what the user wants or requires and what is provided by the information products and services”<ref name="BaileyDevelop83" />
 +
|-
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;" rowspan="3"|Service quality
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Responsiveness (n=4)
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|"Willingness to help customers and provide prompt service”<ref name="PittService95">{{cite journal |title=Service Quality: A Measure of Information Systems Effectiveness |journal=MIS Quarterly |author=Pitt, L.F.; Watson, R.T.; Kavan, C.B. |volume=19 |issue=2 |pages=173–87 |year=1995 |doi=10.2307/249687}}</ref>
 +
|-
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Assurance (n=4)
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|“Knowledge and courtesy of employees and their ability to inspire trust and confidence”<ref name="PittService95" />
 +
|-
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Empathy (n=4)
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|"Caring individualized attention the service provider gives”<ref name="PittService95" />
 +
|-
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Satisfaction
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Evaluation (Measure was dimension) (n=4)
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|“The level of overall user’s satisfaction with HIS”<ref name="Garcia-SmithDevelop13" />
 +
|-
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|System use
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Evaluation (Measure was equal to dimension) (n=1)
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|The number of hours that users use the HIS
 +
|-
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|System usefulness
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Evaluation (Measure was equal to dimension) (n=4)
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|"The degree to which user believes that using the new system would improve his/her job performance”<ref name="VenkateshTech08">{{cite journal |title=Technology Acceptance Model 3 and a Research Agenda on Interventions |journal=Decision Sciences |author=Venkatesh, V.; Bala, H. |volume=39 |issue=2 |pages=273–315 |year=2008 |doi=10.1111/j.1540-5915.2008.00192.x}}</ref>
 +
|-
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Net benefits
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Evaluation (Measure was equal to dimension) (n=18)
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|"As the 'impacts' of HIS have evolved beyond the immediate user, researchers have suggested additional IS impact measures, such as workgroup impacts, inter-organizational and industry impacts, consumer impacts, and societal impacts.”<ref name="DeLoneTheDeLone03" />
 +
|-
 +
|}
 +
|}
 +
 +
===Study settings===
 +
The study was conducted in Ibn-e Sina and Dr. Hejazi Psychiatry Hospital and education center. The case hospital was a 900-bed teaching hospital, which has been in operation for more than 40 years. The hospital is the largest psychiatric hospital in Northeastern Iran. Patients from five neighboring provinces (with an approximate population of eleven million) are referred to this case hospital for professional psychological services. The hospital consists of 16 specialized and sub-specialized departments such as psychiatric emergency, pediatric psychiatry, geriatric psychiatry, drug abuse treatment, and adult psychiatry. Average bed occupancy and bed turnover ratio is 1.01 and 75%, respectively. The length of stay in the hospital is nine days on average.
 +
 +
In 2002, Mashhad University of Medical Sciences implemented a customized HIS named IHIS (Iranian Hospital Information System). Currently, IHIS is implemented in all hospitals affiliated with the university. The IHIS installation of the case hospital covers information systems (IS) for different services, including admission, discharge and transfer, inpatient, outpatient, pharmacy, [[laboratory]], radiology, accounting, and insurance.
 +
 +
===Participants===
 +
Target respondents were users in four departments, including psychiatric departments, administrative and financial departments, nursing management departments, and para-clinics departments. Given the high number of nurses in the psychiatric departments (n=190), a sample size of 64 participants for nurses was calculated. As well, seven secretaries of the psychiatric departments also participated in the study, making a total of 71 participants from the psychiatric department. All users in other departments (administrative and financial departments, nursing management department, and para-clinics departments) were asked to participate in the current study, of whom a total of 61 participants agreed to participate. Thus, the total number of participants for the study was 125 individuals. Empirical data were collected targeting users with over six months of HIS working experience.
 +
 +
The study was approved by the ethical committee of Mashhad University of Medical Sciences before the instrument was officially distributed to all users to protect the rights and privacy of the participants.
 +
 +
The researchers met all users in person and invited them to participate in the study. Questionnaires were provided to users who agreed to participate in the study.
 +
 +
===Statistics===
 +
Summary statistics for characteristics of participants were calculated as frequencies and proportions. The negative responses were reversed. HIS success rate in the case hospital was calculated in two steps. First, the rate of HIS success by each user in evaluation measures, dimensions, and total dimensions of the instrument was determined. This was performed using the following formula, keeping in mind that five was the maximum score for each question on a 1-5 point Likert scale:
 +
 +
: '''Maximum HIS success rate of evaluation measures, dimensions, and total dimensions by each user''' = number of questions * 5
 +
 +
: '''Acquired HIS success rate of evaluation measures, dimensions, and total dimensions by each user''' = sum of the acquired score for each question on a 1-5 point Likert scale by each user
 +
 +
: '''HIS success rate''' = (Acquired HIS success rate/Maximum HIS success rate) * 100
 +
 +
In the following paragraphs, a sample of conducted calculations will be explained. The “Availability” evaluation measure belonged to the system quality dimension. This evaluation measure has six related questions, and users responded to each question on a 1-5 point Likert scale. The maximum possible score for “Availability” by each user was equal to 1 (one user) * 6 (the number of questions) * 5 (the maximum score for each question on a 1-5 point Likert scale) = 30. The HIS success rate for “Availability” by “user one” was 27. Therefore, the HIS success rate for “Availability” by “user one” = (27/30)*100 = 90%.
 +
 +
The dimension of “System quality” consisted of 14 questions. The maximum possible score for system quality by each user was equal to 1 (one user) * 14 (the number of questions) * 5 (the maximum score for each question) = 70. The HIS success rate of "System quality" by “user one” was 51. Therefore, the HIS success rate for system quality by “user one” = (51/70)*100 = 72.9%.
 +
 +
Combining all dimensions of the instrument, we find a total of 61 questions. Accordingly, the maximum possible score of all dimensions by each user was equal to 1 (one user) * 60 (the number of questions) * 5 (the maximum score for each question) = 300. The HIS success rate for total dimensions of the instrument by “user one” was 205. Therefore, the HIS success rate for total dimensions of the instrument by “user one” = (205/300)*100 = 68.3%.
 +
 +
In the second step, the mean and confidence interval of the HIS success rate were determined for evaluation measures, dimensions, and total dimensions of the instrument by four groups of users. These groups include para-clinics departments, administrative and financial departments, nursing management department, and psychiatric departments. An assessment of the normality of data is a necessity for many statistical tests. Hence, to compare HIS success rate among four user groups, the normality of data in each ISSM dimension was assessed. Some researchers suggested the Shapiro-Wilk test as the best option for testing the normality of data.<ref name="GoodhueUnder95">{{cite journal |title=Understanding User Evaluations of Information Systems |journal=Management Science |author=Goodhue, D.L. |volume=41 |issue=12 |pages=1827–1977 |year=1995 |doi=10.1287/mnsc.41.12.1827}}</ref> In our study, the Shapiro-Wilk test was used to assess the normality of data. In the current study, comparisons of the HIS success rate were made among four user groups using ANOVA for normal variables and the Kruskal-Wallis test for non-normal variables. A p-value of less than 0.05 was considered statistically significant.
 +
 +
In this study, two post-hoc tests were chosen to identify significant inter-group differences at p ≥ 0.05. The Tukey honestly significant difference (HSD) test for normal variables and the Mann–Whitney test for non-normal variables was conducted. Data analysis was performed using SPSS, version 11.5 statistical software.
 +
 +
==Results==
 +
===Participants===
 +
In this study, the researchers met with 191 target users, among which 125 individuals agreed to participate in the study. Table 2 demonstrates the characteristics of the participants. Over two-thirds of the participants were women. The age range of users was 22 to 55 years, and most of the participants were aged 20-30. Only 2% of participants were without a university education. More than 77% of the participants had a two- or four-year university degree. 61% of participants had 5-15 years of work experience. 68% of the users had an ICDL certificate. 80% of the users had over three years of HIS experience, and 68% of them had more than six years of computer experience.
 +
 +
{|
 +
| STYLE="vertical-align:top;"|
 +
{| class="wikitable" border="1" cellpadding="5" cellspacing="0" width="80%"
 +
|-
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;" colspan="3"|'''Table 2.''' Participants' characteristics
 +
|-
 +
  ! style="padding-left:10px; padding-right:10px;"|User's characteristics
 +
  ! style="padding-left:10px; padding-right:10px;"|Subgroups
 +
  ! style="padding-left:10px; padding-right:10px;"|Frequency (%)
 +
|-
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;" rowspan="2"|Sex
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Male
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|48 (38.4)
 +
|-
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Female
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|77 (61.6)
 +
|-
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;" rowspan="4"|Age (years)
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|20–29
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|20 (16.0)
 +
|-
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|30–39
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|73 (58.4)
 +
|-
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|40–49
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|30 (24.0)
 +
|-
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|≥ 50
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|2 (1.6)
 +
|-
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;" rowspan="4"|Education level
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|High school diploma
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|2 (1.6)
 +
|-
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Associate's
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|2 (1.6)
 +
|-
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Bachelor's
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|94 (75.2)
 +
|-
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Master's
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|27 (21.6)
 +
|-
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;" rowspan="4"|Working hours per month
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|< Regular + 20 overtime hours
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|41 (20.8)
 +
|-
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Regular + 20–60 overtime hours
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|44 (35.2)
 +
|-
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Regular + 60–120 overtime hours
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|30 (24.0)
 +
|-
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|> Regular + 120 overtime hours
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|10 (8.0)
 +
|-
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;" rowspan="6"|Work experience (years)
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|< 5
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|23 (18.4)
 +
|-
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|5–9
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|35 (28.0)
 +
|-
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|10–14
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|41 (32.8)
 +
|-
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|15–19
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|13 (10.4)
 +
|-
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|20–24
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|9 (7.2)
 +
|-
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|≥ 25
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|4 (3.2)
 +
|-
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;" rowspan="2"|ICDL certification
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Yes
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|97 (77.6)
 +
|-
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|No
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|28 (22.4)
 +
|-
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;" rowspan="3"|Computer experience (years)
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|< 1
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|15 (12.0)
 +
|-
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|1–3
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|9 (7.2)
 +
|-
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|> 3
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|101 (80.8)
 +
|-
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;" rowspan="4"|HIS experience (years)
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|< 1
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|3 (2.4)
 +
|-
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|1–3
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|10 (8.0)
 +
|-
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|4–6
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|14 (11.2)
 +
|-
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|> 6
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|98 (78.4)
 +
|-
 +
|}
 +
|}
 +
 +
===Reliability and validity===
 +
The instrument was validated by an expert panel with CVI at 0.85 and CVR at 0.86. The overall Cronbach’s alpha value of the instrument was determined as 0.93, representing high reliability. This value was between 0.476 and 0.943 for different instrument dimensions, as shown in Table 3. The value of Cronbach’s alpha was satisfactory among the five dimensions of system quality, information quality, service quality, usefulness, and net benefits. The value of Cronbach’s alpha for the satisfaction dimension was low, and its value was 47.0. Because the dimension of “system use” had just one question, its Cronbach’s alpha was not calculated. The Pearson correlation coefficient showed significant positive relationships among the investigated dimensions (p-value: 0.01). Pearson correlation coefficient was between 0.197 and 0.707 for different dimensions (Table 3).
 +
 +
{|
 +
| STYLE="vertical-align:top;"|
 +
{| class="wikitable" border="1" cellpadding="5" cellspacing="0" width="100%"
 +
|-
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;" colspan="8"|'''Table 3.''' Correlations among all dimensions (n = 125)<br />* Correlation is significant at the 0.05 significance level (2-tailed)<br />** Correlation is significant at the 0.01 significance level (2-tailed)
 +
|-
 +
  ! style="padding-left:10px; padding-right:10px;"|Dimensions (Cronbach’s alpha)
 +
  ! style="padding-left:10px; padding-right:10px;"|System quality
 +
  ! style="padding-left:10px; padding-right:10px;"|Information quality
 +
  ! style="padding-left:10px; padding-right:10px;"|Service quality
 +
  ! style="padding-left:10px; padding-right:10px;"|System use
 +
  ! style="padding-left:10px; padding-right:10px;"|Satisfaction
 +
  ! style="padding-left:10px; padding-right:10px;"|Usefulness
 +
  ! style="padding-left:10px; padding-right:10px;"|Net benefits
 +
|-
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|System quality (0.719)
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|1
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.401**
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.255**
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.450**
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.319**
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.270**
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.348**
 +
|-
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Information quality (0.611)
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|1
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.454**
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.315**
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.471**
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.495**
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.454**
 +
|-
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Service quality (0.785)
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|1
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.197*
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.329**
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.285**
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.352**
 +
|-
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Satisfaction (0.476)
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|1
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.423**
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.444**
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.411**
 +
|-
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|System use (not calculated)
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|1
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.660**
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.632**
 +
|-
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Usefulness (0.926)
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|1
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.707**
 +
|-
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Net benefits (0.943)
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|1
 +
|-
 +
|}
 +
|}
 +
 +
===HIS success rate===
 +
 +
{|
 +
| STYLE="vertical-align:top;"|
 +
{| class="wikitable" border="1" cellpadding="5" cellspacing="0" width="100%"
 +
|-
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;" colspan="6"|'''Table 4.''' Mean success rate of evaluation dimensions<br />The significant results within each group of users are indicated by letters <sup>a</sup> and <sup>b</sup>; values not sharing a common letter differ significantly (P < 0.05).
 +
|-
 +
  ! style="padding-left:10px; padding-right:10px;" rowspan="2"|Users/Dimensions
 +
  ! style="padding-left:10px; padding-right:10px;" colspan="5"|Mean (95% confidence interval)
 +
|-
 +
  ! style="padding-left:10px; padding-right:10px;"|Financial and administrative departments
 +
  ! style="padding-left:10px; padding-right:10px;"|Psychiatric departments
 +
  ! style="padding-left:10px; padding-right:10px;"|Nursing management department
 +
  ! style="padding-left:10px; padding-right:10px;"|Para-clinical department
 +
  ! style="padding-left:10px; padding-right:10px;"|All users
 +
|-
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|System quality
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.74 (0.71, 0.77)
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.65 (0.63, 0.67)
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.71 (0.65, 0.78)<sup>b</sup>
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.64 (0.59, 0.70)<sup>b</sup>
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.67 (0.65, 0.69)
 +
|-
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Information quality
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.65 (0.61, 0.69)
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.62 (0.60, 0.64)
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.62 (0.56, 0.68)
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.62 (0.58, 0.66)
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.63 (0.61, 0.64)
 +
|-
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Service quality
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.62 (0.56, 0.67)
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.63 (0.61, 0.66)
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.62 (0.56, 0.69)
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.65 (0.62, 0.69)
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.63 (0.61, 0.65)
 +
|-
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|System use
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.63 (0.47, 0.79)<sup>a</sup>
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.30 (0.25, 0.35)<sup>b</sup>
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.27 (0.16, 0.37)<sup>b</sup>
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.41 (0.29, 0.54)<sup>b</sup>
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.37 (0.32, 0.41)
 +
|-
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Satisfaction
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.68 (0.63, 0.73)
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.63 (0.60, 0.66)
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.68 (0.61, 0.75)
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.66 (0.62, 0.71)
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.65 (0.63, 0.67)
 +
|-
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Usefulness
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.78 (0.71, 0.85)
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.71 (0.67, 0.75)
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.77 (0.67, 0.86)
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.76 (0.72, 0.80)
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.74 (0.71, 0.76)
 +
|-
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Net benefits
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.68 (0.62, 0.74)
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.64 (0.61, 0.68)
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.70 (0.60, 0.79)
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.66 (0.61, 0.71)
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.66 (0.63, 0.68)
 +
|-
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|'''Total HIS success'''
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.69 (0.65, 0.72)
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.64 (0.62, 0.66)
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.67 (0.61, 0.74)
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.65 (0.62, 0.69)
 +
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.65 (0.64, 0.67)
 +
|-
 +
|}
 +
|}
  
 
==References==
 
==References==
Line 46: Line 429:
  
 
==Notes==
 
==Notes==
This presentation is faithful to the original, with only a few minor changes to presentation, spelling, and grammar. We also added PMCID and DOI when they were missing from the original reference.
+
This presentation is faithful to the original, with only a few minor changes to presentation, spelling, and grammar. We also added PMCID and DOI when they were missing from the original reference. The original age ranges, work experience ranges, and HIS experience ranges in Table 2 overlapped inappropriately; they have been updated here to what presumably the authors intended. The original erroneously refers to Table 3 as Table 4 twice; that is updated in this version.
  
 
<!--Place all category tags here-->
 
<!--Place all category tags here-->

Revision as of 19:50, 11 June 2019

Full article title Determining the hospital information system (HIS) success rate: Development of a new instrument and case study
Journal Open Access Macedonian Journal of Medical Sciences
Author(s) Ebnehoseini, Zahara; Tabesh, Hamed; Deldar, Kolsoum; Mostafavi, Sayyed M.; Tara, Mahmood
Author affiliation(s) Mashhad University of Medical Sciences, Shahroud University of Medical Sciences
Primary contact Email: TaraM at mums dot ac dot ir
Year published 2019
Volume and issue 7(9)
Page(s) 1407–14
DOI 10.3889/oamjms.2019.294
ISSN 1857-9655
Distribution license Creative Commons Attribution-NonCommercial 4.0 International
Website https://www.id-press.eu/mjms/article/view/2647
Download https://www.id-press.eu/mjms/article/view/oamjms.2019.294/3327 (PDF)

Abstract

Background: A hospital information system (HIS) is a type of health information system which is widely used in clinical settings. Determining the success rate of a HIS is an ongoing area of research since its implications are of interest for researchers, physicians, and managers.

Aim: In the present study, we develop a novel instrument to measure HIS success rate based on users’ viewpoints in a teaching hospital.

Methods: The study was conducted in Ibn-e Sina and Dr. Hejazi Psychiatry Hospital and education center in Mashhad, Iran. The instrument for data collection was a self-administered structured questionnaire based on the information systems success model (ISSM), covering seven dimensions, which includes system quality, information quality, service quality, system use, usefulness, satisfaction, and net benefits. The verification of content validity was carried out by an expert panel. The internal consistency of dimensions was measured by Cronbach’s alpha. Pearson’s correlation coefficient was calculated to evaluate the significance of associations between dimensions. The HIS success rate on users’ viewpoints was determined.

Results: A total of 125 users participated in the study. The instrument was validated by an expert panel with the content validity index (CVI) at 0.85 and content validity ratio (CVR) at 0.86. The overall Cronbach’s alpha value of the instrument was 0.93. The Pearson correlation coefficient showed significant positive relationships among the investigated dimensions. On average, HIS success rate in the hospital under study was 65% (CI: 64%, 67%). The dimensions of “usefulness,” “system quality,” and “net benefits” showed the highest rates of success, respectively.

Conclusion: The instrument used in this study can be adopted for HIS evaluation in future studies. In the current study, a method was developed to determine the HIS success rate based on users’ viewpoints. This method allows for the comparison of HIS success rates in various hospitals. As well, our findings underscore the viewpoints of HIS users in a developing country.

Keywords: electronic health record, evaluation, hospital information systems, information system success model, ISSM

Introduction

Hospitals have implemented health information systems to provide timely and accurate information, thus fulfilling managerial needs and improving healthcare effectiveness and efficiency at a reasonable cost.[1]

The hospital information system (HIS) is one of the health information systems which has been extensively utilized. There are major concerns regarding the quality of these systems; therefore, it is vital to continuously evaluate the HIS. As well, HIS implementation requires a great deal of investment. For example, the Health Information Technology for Economic and Clinical Health Act (HITECH) of 2009 mandates about $10 million for each hospital in the United States.[2] Therefore, post-implementation evaluation of HIS success is of critical importance. Determining HIS success rate is an ongoing area of research since its implications are of interest not only for researchers but also to physicians and managers. The results of the evaluation can reveal the value of a HIS and provide essential information for subsequent decisions.[3]

There are many existing frameworks to evaluate health information systems. The information systems success model (ISSM) is one of the most well-known and valid frameworks to assess these systems.[4] ISSM was developed by DeLone and Mclean in 1990[5] and was subsequently updated in 2003.[6] ISSM is comprised of seven dimensions, including system quality, information quality, service quality, system use, usefulness, satisfaction, and net benefits.[6] ISSM focuses on the assessment of technological aspects of information systems based on users’ viewpoints and explains the influence of technology on system use as well as users’ satisfaction. The success of information systems depends on the interaction of different ISSM dimensions.[7] Some of the studies which have deployed this framework to assess HIS include works by Tilahun and Fritz[4], Sicotte et al.[8], Otieno et al.[9], Ojo[3], Messeri et al.[10], and Aggelidis et al.[11]

In 2014, the Iranian Ministry of Health and Medical Education mandated that public hospitals transfer patients’ medical records to SEPAS (Iran’s electronic health record [EHR] project). This led to a rapid growth in the implementation of a HIS in hospitals across the country. Following the implementation of EHRs and HISs in Saudi Arabia[12], Korea[2], and Taiwan[13][14], the success of these systems was evaluated. In the case of Iran, a developing country, to the best of our knowledge there have been very few studies which have evaluated the success of a HIS. The studies by Tavakoli et al.[15], Abdekhoda et al.[16], and Saghaeiannejad-Isfahani et al.[17] offer some examples to this end. Nevertheless, these studies did not collectively provide conclusive evidence on the success of HISs. As such, there is a need for additional studies to gather further evidence on the effectiveness and efficiency of the HIS in Iran.

In the present study, we report on the development of an instrument to evaluate the rate of HIS success based on ISSM, which can be used to compare HIS success rates in different hospitals and clinical settings. Furthermore, in a case study, we investigate the success rate of HIS based on users’ viewpoints in a teaching hospital in Iran.

Methodology

This was a descriptive cross-sectional study.

Instrument

A self-administrated structured questionnaire was developed based on ISSM to collect data. The questionnaire comprised of seven dimensions, including system quality, information quality, service quality, system use, usefulness, satisfaction, and net benefits. Each dimension encompasses several evaluation measures. The questions were designed based on the ISSM evaluation measures and dimensions.

Verification of the content validity was carried out by an expert panel. The panel consisted of nine medical informatics experts. Expert validity was measured using a content validity index (CVI) and content validity ratio (CVR). The internal consistency of each dimension was measured by Cronbach’s alpha. Pearson’s correlation was calculated to evaluate the significance of associations between dimensions.

The final draft of the instrument consisted of two sections. The first section includes users’ characteristics such as age, gender, education level, working hours per month, ICDL certification, work experience, computer experience, and HIS experience (n=8 questions). The second section includes 60 questions on a five-point Likert scale ranging from 1 (completely disagree) to 5 (completely agree), which covers 16 evaluation measures in seven dimensions (Table 1).

Table 1. Dimensions and evaluation measures
Dimensions Evaluation Measure
(number of questions)
Definition
System quality Adaptability (n=1) “Data from different sources can be consolidated or compared without inconsistencies.”[18]
Reliability (n=1) “The dependability and consistency of access and uptime of systems”[18][19]
Availability (n=6) Access to the computer resource and infrastructure where and when it is needed
Usability (n=6) “The capability in human functional terms to be used easily and effectively by the specified range of users, given specified training and user support, to fulfill the specified range of tasks, within the specified range of environmental scenarios”[20]
Information quality Security (n=3) “The safeguarding of data from misappropriation or unauthorized alteration or loss”[21]
Ease of understanding (n=3) “Ease of determining what a data element on a report or file means, or what is excluded or included in calculating it”[18]
Completeness (n=2) “The comprehensive of the output information content”[21]
Personalization (n=1) Ability to personalize HIS for users
Relevance (n=1) “The degree of congruence between what the user wants or requires and what is provided by the information products and services”[21]
Service quality Responsiveness (n=4) "Willingness to help customers and provide prompt service”[22]
Assurance (n=4) “Knowledge and courtesy of employees and their ability to inspire trust and confidence”[22]
Empathy (n=4) "Caring individualized attention the service provider gives”[22]
Satisfaction Evaluation (Measure was dimension) (n=4) “The level of overall user’s satisfaction with HIS”[19]
System use Evaluation (Measure was equal to dimension) (n=1) The number of hours that users use the HIS
System usefulness Evaluation (Measure was equal to dimension) (n=4) "The degree to which user believes that using the new system would improve his/her job performance”[23]
Net benefits Evaluation (Measure was equal to dimension) (n=18) "As the 'impacts' of HIS have evolved beyond the immediate user, researchers have suggested additional IS impact measures, such as workgroup impacts, inter-organizational and industry impacts, consumer impacts, and societal impacts.”[6]

Study settings

The study was conducted in Ibn-e Sina and Dr. Hejazi Psychiatry Hospital and education center. The case hospital was a 900-bed teaching hospital, which has been in operation for more than 40 years. The hospital is the largest psychiatric hospital in Northeastern Iran. Patients from five neighboring provinces (with an approximate population of eleven million) are referred to this case hospital for professional psychological services. The hospital consists of 16 specialized and sub-specialized departments such as psychiatric emergency, pediatric psychiatry, geriatric psychiatry, drug abuse treatment, and adult psychiatry. Average bed occupancy and bed turnover ratio is 1.01 and 75%, respectively. The length of stay in the hospital is nine days on average.

In 2002, Mashhad University of Medical Sciences implemented a customized HIS named IHIS (Iranian Hospital Information System). Currently, IHIS is implemented in all hospitals affiliated with the university. The IHIS installation of the case hospital covers information systems (IS) for different services, including admission, discharge and transfer, inpatient, outpatient, pharmacy, laboratory, radiology, accounting, and insurance.

Participants

Target respondents were users in four departments, including psychiatric departments, administrative and financial departments, nursing management departments, and para-clinics departments. Given the high number of nurses in the psychiatric departments (n=190), a sample size of 64 participants for nurses was calculated. As well, seven secretaries of the psychiatric departments also participated in the study, making a total of 71 participants from the psychiatric department. All users in other departments (administrative and financial departments, nursing management department, and para-clinics departments) were asked to participate in the current study, of whom a total of 61 participants agreed to participate. Thus, the total number of participants for the study was 125 individuals. Empirical data were collected targeting users with over six months of HIS working experience.

The study was approved by the ethical committee of Mashhad University of Medical Sciences before the instrument was officially distributed to all users to protect the rights and privacy of the participants.

The researchers met all users in person and invited them to participate in the study. Questionnaires were provided to users who agreed to participate in the study.

Statistics

Summary statistics for characteristics of participants were calculated as frequencies and proportions. The negative responses were reversed. HIS success rate in the case hospital was calculated in two steps. First, the rate of HIS success by each user in evaluation measures, dimensions, and total dimensions of the instrument was determined. This was performed using the following formula, keeping in mind that five was the maximum score for each question on a 1-5 point Likert scale:

Maximum HIS success rate of evaluation measures, dimensions, and total dimensions by each user = number of questions * 5
Acquired HIS success rate of evaluation measures, dimensions, and total dimensions by each user = sum of the acquired score for each question on a 1-5 point Likert scale by each user
HIS success rate = (Acquired HIS success rate/Maximum HIS success rate) * 100

In the following paragraphs, a sample of conducted calculations will be explained. The “Availability” evaluation measure belonged to the system quality dimension. This evaluation measure has six related questions, and users responded to each question on a 1-5 point Likert scale. The maximum possible score for “Availability” by each user was equal to 1 (one user) * 6 (the number of questions) * 5 (the maximum score for each question on a 1-5 point Likert scale) = 30. The HIS success rate for “Availability” by “user one” was 27. Therefore, the HIS success rate for “Availability” by “user one” = (27/30)*100 = 90%.

The dimension of “System quality” consisted of 14 questions. The maximum possible score for system quality by each user was equal to 1 (one user) * 14 (the number of questions) * 5 (the maximum score for each question) = 70. The HIS success rate of "System quality" by “user one” was 51. Therefore, the HIS success rate for system quality by “user one” = (51/70)*100 = 72.9%.

Combining all dimensions of the instrument, we find a total of 61 questions. Accordingly, the maximum possible score of all dimensions by each user was equal to 1 (one user) * 60 (the number of questions) * 5 (the maximum score for each question) = 300. The HIS success rate for total dimensions of the instrument by “user one” was 205. Therefore, the HIS success rate for total dimensions of the instrument by “user one” = (205/300)*100 = 68.3%.

In the second step, the mean and confidence interval of the HIS success rate were determined for evaluation measures, dimensions, and total dimensions of the instrument by four groups of users. These groups include para-clinics departments, administrative and financial departments, nursing management department, and psychiatric departments. An assessment of the normality of data is a necessity for many statistical tests. Hence, to compare HIS success rate among four user groups, the normality of data in each ISSM dimension was assessed. Some researchers suggested the Shapiro-Wilk test as the best option for testing the normality of data.[24] In our study, the Shapiro-Wilk test was used to assess the normality of data. In the current study, comparisons of the HIS success rate were made among four user groups using ANOVA for normal variables and the Kruskal-Wallis test for non-normal variables. A p-value of less than 0.05 was considered statistically significant.

In this study, two post-hoc tests were chosen to identify significant inter-group differences at p ≥ 0.05. The Tukey honestly significant difference (HSD) test for normal variables and the Mann–Whitney test for non-normal variables was conducted. Data analysis was performed using SPSS, version 11.5 statistical software.

Results

Participants

In this study, the researchers met with 191 target users, among which 125 individuals agreed to participate in the study. Table 2 demonstrates the characteristics of the participants. Over two-thirds of the participants were women. The age range of users was 22 to 55 years, and most of the participants were aged 20-30. Only 2% of participants were without a university education. More than 77% of the participants had a two- or four-year university degree. 61% of participants had 5-15 years of work experience. 68% of the users had an ICDL certificate. 80% of the users had over three years of HIS experience, and 68% of them had more than six years of computer experience.

Table 2. Participants' characteristics
User's characteristics Subgroups Frequency (%)
Sex Male 48 (38.4)
Female 77 (61.6)
Age (years) 20–29 20 (16.0)
30–39 73 (58.4)
40–49 30 (24.0)
≥ 50 2 (1.6)
Education level High school diploma 2 (1.6)
Associate's 2 (1.6)
Bachelor's 94 (75.2)
Master's 27 (21.6)
Working hours per month < Regular + 20 overtime hours 41 (20.8)
Regular + 20–60 overtime hours 44 (35.2)
Regular + 60–120 overtime hours 30 (24.0)
> Regular + 120 overtime hours 10 (8.0)
Work experience (years) < 5 23 (18.4)
5–9 35 (28.0)
10–14 41 (32.8)
15–19 13 (10.4)
20–24 9 (7.2)
≥ 25 4 (3.2)
ICDL certification Yes 97 (77.6)
No 28 (22.4)
Computer experience (years) < 1 15 (12.0)
1–3 9 (7.2)
> 3 101 (80.8)
HIS experience (years) < 1 3 (2.4)
1–3 10 (8.0)
4–6 14 (11.2)
> 6 98 (78.4)

Reliability and validity

The instrument was validated by an expert panel with CVI at 0.85 and CVR at 0.86. The overall Cronbach’s alpha value of the instrument was determined as 0.93, representing high reliability. This value was between 0.476 and 0.943 for different instrument dimensions, as shown in Table 3. The value of Cronbach’s alpha was satisfactory among the five dimensions of system quality, information quality, service quality, usefulness, and net benefits. The value of Cronbach’s alpha for the satisfaction dimension was low, and its value was 47.0. Because the dimension of “system use” had just one question, its Cronbach’s alpha was not calculated. The Pearson correlation coefficient showed significant positive relationships among the investigated dimensions (p-value: 0.01). Pearson correlation coefficient was between 0.197 and 0.707 for different dimensions (Table 3).

Table 3. Correlations among all dimensions (n = 125)
* Correlation is significant at the 0.05 significance level (2-tailed)
** Correlation is significant at the 0.01 significance level (2-tailed)
Dimensions (Cronbach’s alpha) System quality Information quality Service quality System use Satisfaction Usefulness Net benefits
System quality (0.719) 1 0.401** 0.255** 0.450** 0.319** 0.270** 0.348**
Information quality (0.611) 1 0.454** 0.315** 0.471** 0.495** 0.454**
Service quality (0.785) 1 0.197* 0.329** 0.285** 0.352**
Satisfaction (0.476) 1 0.423** 0.444** 0.411**
System use (not calculated) 1 0.660** 0.632**
Usefulness (0.926) 1 0.707**
Net benefits (0.943) 1

HIS success rate

Table 4. Mean success rate of evaluation dimensions
The significant results within each group of users are indicated by letters a and b; values not sharing a common letter differ significantly (P < 0.05).
Users/Dimensions Mean (95% confidence interval)
Financial and administrative departments Psychiatric departments Nursing management department Para-clinical department All users
System quality 0.74 (0.71, 0.77) 0.65 (0.63, 0.67) 0.71 (0.65, 0.78)b 0.64 (0.59, 0.70)b 0.67 (0.65, 0.69)
Information quality 0.65 (0.61, 0.69) 0.62 (0.60, 0.64) 0.62 (0.56, 0.68) 0.62 (0.58, 0.66) 0.63 (0.61, 0.64)
Service quality 0.62 (0.56, 0.67) 0.63 (0.61, 0.66) 0.62 (0.56, 0.69) 0.65 (0.62, 0.69) 0.63 (0.61, 0.65)
System use 0.63 (0.47, 0.79)a 0.30 (0.25, 0.35)b 0.27 (0.16, 0.37)b 0.41 (0.29, 0.54)b 0.37 (0.32, 0.41)
Satisfaction 0.68 (0.63, 0.73) 0.63 (0.60, 0.66) 0.68 (0.61, 0.75) 0.66 (0.62, 0.71) 0.65 (0.63, 0.67)
Usefulness 0.78 (0.71, 0.85) 0.71 (0.67, 0.75) 0.77 (0.67, 0.86) 0.76 (0.72, 0.80) 0.74 (0.71, 0.76)
Net benefits 0.68 (0.62, 0.74) 0.64 (0.61, 0.68) 0.70 (0.60, 0.79) 0.66 (0.61, 0.71) 0.66 (0.63, 0.68)
Total HIS success 0.69 (0.65, 0.72) 0.64 (0.62, 0.66) 0.67 (0.61, 0.74) 0.65 (0.62, 0.69) 0.65 (0.64, 0.67)

References

  1. Chen, R.F.; Hsiao, J.L. (2012). "An investigation on physicians' acceptance of hospital information systems: A case study". International Journal of Medical Informatics 81 (12): 810–20. doi:10.1016/j.ijmedinf.2012.05.003. PMID 22652011. 
  2. 2.0 2.1 Iqbal, U.; Ho, C.H.; Li, Y.C. et al. (2013). "The relationship between usage intention and adoption of electronic health records at primary care clinics". Computer Methods and Programs in Biomedicine 112 (3): 731–7. doi:10.1016/j.cmpb.2013.09.001. PMID 24091088. 
  3. 3.0 3.1 Ojo, A.I. (2017). "Validation of the DeLone and McLean Information Systems Success Model". Healthcare Informatics Research 23 (1): 60–66. doi:10.4258/hir.2017.23.1.60. PMC PMC5334133. PMID 28261532. http://www.pubmedcentral.nih.gov/articlerender.fcgi?tool=pmcentrez&artid=PMC5334133. 
  4. 4.0 4.1 Tilahun, B.; Fritz, F. (2015). "Modeling antecedents of electronic medical record system implementation success in low-resource setting hospitals". BMC Medical Informatics and Decision Making 15: 61. doi:10.1186/s12911-015-0192-0. PMC PMC4522063. PMID 26231051. http://www.pubmedcentral.nih.gov/articlerender.fcgi?tool=pmcentrez&artid=PMC4522063. 
  5. DeLone, W.H.; McLean, E.R. (1992). "Information Systems Success: The Quest for the Dependent Variable". Information Systems Research 3 (1): 1–95. doi:10.1287/isre.3.1.60. 
  6. 6.0 6.1 6.2 DeLone, W.H.; McLean, E.R. (2003). "The DeLone and McLean Model of Information Systems Success: A Ten-Year Update". Journal of Management Information Systems 19 (4): 9–30. doi:10.1080/07421222.2003.11045748. 
  7. Otieno, G.O.; Hinako, T.; Motohiro, A. et al. (2008). "Measuring effectiveness of electronic medical records systems: Towards building a composite index for benchmarking hospitals". International Journal of Medical Informatics 77 (10): 657–69. doi:10.1016/j.ijmedinf.2008.01.002. PMID 18313352. 
  8. Sicotte, C.; Paré, G.; Bini, K.K. et al. (2010). "Virtual organization of hospital medical imaging: A user satisfaction survey". Journal of Digital Imaging 23 (6): 689–700. doi:10.1007/s10278-009-9220-x. PMC PMC3046687. PMID 19588196. http://www.pubmedcentral.nih.gov/articlerender.fcgi?tool=pmcentrez&artid=PMC3046687. 
  9. Otieno, O.G.; Toyama, H.; Asonuma, M. et al. (2007). "Nurses' views on the use, quality and user satisfaction with electronic medical records: Questionnaire development". Journal of Advanced Nursing 60 (2): 209–19. doi:10.1111/j.1365-2648.2007.04384.x. PMID 17877568. 
  10. Messeri, P.; Khan, S.; Millery, M. et al. (2013). "An information systems model of the determinants of electronic health record use". Applied Clinical Informatics 4 (2): 185–200. doi:10.4338/ACI-2013-01-RA-0005. PMC PMC3716425. PMID 23874357. http://www.pubmedcentral.nih.gov/articlerender.fcgi?tool=pmcentrez&artid=PMC3716425. 
  11. Aggelidis, V.P.; Chatzoglou, P.D. (2012). "Hospital information systems: Measuring end user computing satisfaction (EUCS)". Journal of Biomedical Informatics 45 (3): 566–79. doi:10.1016/j.jbi.2012.02.009. PMID 22426283. 
  12. Alharthi, H.; Youssef, A.; Radwan, S. et al. (2014). "Physician satisfaction with electronic medical records in a major Saudi Government hospital". Journal of Taibah University Medical Sciences 9 (3): 213–18. doi:10.1016/j.jtumed.2014.01.004. 
  13. Wang, J.Y.; Ho, H.Y.; Chen, J.D. et al. (2015). "Attitudes toward inter-hospital electronic patient record exchange: Discrepancies among physicians, medical record staff, and patients". BMC Health Services Research 15: 264. doi:10.1186/s12913-015-0896-y. PMC PMC4499194. PMID 26162748. http://www.pubmedcentral.nih.gov/articlerender.fcgi?tool=pmcentrez&artid=PMC4499194. 
  14. Lu, C.H.; Hsiao, J.L.; Chen, R.F. (2012). "Factors determining nurse acceptance of hospital information systems". Computers, Informatics, Nursing 30 (5): 257–64. doi:10.1097/NCN.0b013e318224b4cf. PMID 22228251. 
  15. Tavakoli, N.; Jahanbakhsh, M.; Shahin, A. et al. (2013). "Electronic medical record in central polyclinic of isfahan oil industry: a case study based on technology acceptance model". Acta Informatica Medica 21 (1): 23–5. doi:10.5455/AIM.2012.21.23-25. PMC PMC3610586. PMID 23572857. http://www.pubmedcentral.nih.gov/articlerender.fcgi?tool=pmcentrez&artid=PMC3610586. 
  16. Abdekhoda, M.; Ahmadi, M.; Gohari, M. et al. (2015). "The effects of organizational contextual factors on physicians' attitude toward adoption of Electronic Medical Records". Journal of Biomedical Informatics 53: 174–9. doi:10.1016/j.jbi.2014.10.008. PMID 25445481. 
  17. Saghaeiannejad-Isfahani, S.; Jahanbakhsh, M.; Habibi, M. et al. (2014). "A Survey on the Users' Satisfaction with the Hospital Information Systems (HISs) based on DeLone and McLean's Model in the Medical-Teaching Hospitals in Isfahan City". Acta Informatica Medica 22 (3): 179-82. doi:10.5455/aim.2014.22.179-182. PMC PMC4130683. PMID 25132711. http://www.pubmedcentral.nih.gov/articlerender.fcgi?tool=pmcentrez&artid=PMC4130683. 
  18. 18.0 18.1 18.2 Goodhue, D.L.; Thompson, R.L. (1995). "Task-Technology Fit and Individual Performance". MIS Quarterly 19 (2): 213–36. doi:10.2307/249689. 
  19. 19.0 19.1 Garcia-Smith, D.; Effken, J.A. (2013). "Development and initial evaluation of the Clinical Information Systems Success Model (CISSM)". International Journal of Medical Informatics 82 (6): 539–52. doi:10.1016/j.ijmedinf.2013.01.011. PMID 23497819. 
  20. Shackel, B. (1991). "Usability—Context, framework, definition, design and evaluation". Human Factors for Informatics Usability. Cambridge University Press. pp. 21–37. ISBN 0521365708. 
  21. 21.0 21.1 21.2 Bailey, J.E.; Pearson, S.W. (1983). "Development of a Tool for Measuring and Analyzing Computer User Satisfaction". Management Science 29 (5): 519–637. doi:10.1287/mnsc.29.5.530. 
  22. 22.0 22.1 22.2 Pitt, L.F.; Watson, R.T.; Kavan, C.B. (1995). "Service Quality: A Measure of Information Systems Effectiveness". MIS Quarterly 19 (2): 173–87. doi:10.2307/249687. 
  23. Venkatesh, V.; Bala, H. (2008). "Technology Acceptance Model 3 and a Research Agenda on Interventions". Decision Sciences 39 (2): 273–315. doi:10.1111/j.1540-5915.2008.00192.x. 
  24. Goodhue, D.L. (1995). "Understanding User Evaluations of Information Systems". Management Science 41 (12): 1827–1977. doi:10.1287/mnsc.41.12.1827. 

Notes

This presentation is faithful to the original, with only a few minor changes to presentation, spelling, and grammar. We also added PMCID and DOI when they were missing from the original reference. The original age ranges, work experience ranges, and HIS experience ranges in Table 2 overlapped inappropriately; they have been updated here to what presumably the authors intended. The original erroneously refers to Table 3 as Table 4 twice; that is updated in this version.