Journal:Implementing a novel quality improvement-based approach to data quality monitoring and enhancement in a multipurpose clinical registry

Full article title	Implementing a novel quality improvement-based approach to data quality monitoring and enhancement in a multipurpose clinical registry
Journal	eGEMs: The Journal for Electronic Health Data and Methods
Author(s)	Pratt, J.; Jeffers, D.; King, E.C.; Kappelman, M.D.; Collins, J.; Margolis, P.; Bron, H.; Bass, J.A.; Bassett, M.D.; Beasley, G.L.; Benkov, K.J.; Bornstein, J.A.; Cabrera, J.M.; Crandall, W.; Dancel, L.D.; Garin-Laflam, M.P.; Grunow, J.E.; Hirsch, B.Z.; Hoffenberg, E.; Israel, E.; Jester, T.W.; Kiparissi, F.; Lakhole, A.; Lapsia, S.P.; Minar, P.; Navarro, F.A.; Neef, H.; Park, K.T.; Pashankar, D.S.; Patel, A.S.; Pineiro, V.M.; Samson, C.M.; Sandberg, K.C.; Steiner, S.J.; Strople, J.A.; Sudel, B.; Sullivan, J.S.; Suskind, D.L.; Uppal, V.; Wali, P.D.
Author affiliation(s)	Various (see the original for all affiliations)
Primary contact	Email: eileen dot king at cchmc dot org
Year published	2019
Volume and issue	7(1)
Page(s)	51
DOI	10.5334/egems.262
ISSN	2327-9214
Distribution license	Creative Commons Attribution 4.0 International
Website	https://egems.academyhealth.org/articles/10.5334/egems.262/
Download	https://egems.academyhealth.org/articles/10.5334/egems.262/galley/434/download/ (PDF)

Abstract

Objective: To implement a quality improvement-based system to measure and improve data quality in an observational clinical registry to support a learning healthcare system.

Data source: ImproveCareNow Network registry, which as of September 2019 contained data from 314,250 visits of 43,305 pediatric inflammatory bowel disease (IBD) patients at 109 participating care centers.

Study design: The impact of data quality improvement support to care centers was evaluated using statistical process control methodology. Data quality measures were defined, performance feedback of those measures using statistical process control charts was implemented, and reports that identified data items not following data quality checks were developed to enable centers to monitor and improve the quality of their data.

Principal findings: There was a pattern of improvement across measures of data quality. The proportion of visits with complete critical data increased from 72 percent to 82 percent. The percent of registered patients improved from 59 percent to 83 percent. Of three additional measures of data consistency and timeliness, one improved performance from 42 percent to 63 percent. Performance declined on one measure due to changes in network documentation practices and maturation. There was variation among care centers in data quality.

Conclusions: A quality improvement based approach to data quality monitoring and improvement is feasible and effective.

Keywords: quality improvement, data quality, registry

Introduction

There is growing interest in the potential for clinical registries that can simultaneously support clinical care, quality improvement (QI), and research. This multi-purpose model is consistent with the Institute of Medicine’s (IOM’s) vision of a learning health system which “draws research closer to clinical practice by building knowledge development and application into each stage of the health care delivery process.” Gliklich et al. define a registry as “an organized system that uses observational study methods to collect uniform data (clinical and other) to evaluate specified outcomes for a population defined by a particular disease, condition, or exposure, and that serves one or more predetermined scientific, clinical, or policy purposes.” Most pediatric chronic illnesses meet the National Institutes of Health's (NIH) definition for rare disease, and as such, multi-center registries are especially important to study and improve care for children with chronic diseases. Some multi-center networks are beginning to adopt principles of open science, or network-based production, to foster collaborative improvement, research, data sharing, and innovation. In this setting, the registry functions not only to provide access to condition-specific information in a uniform way to support clinical care but also to support QI and research to improve patient outcomes.

The challenges and opportunities in managing data from multi-purpose clinical registries that are used for care, QI, and research are distinct from those that arise in the management of data collected specifically for study purposes, particularly clinical trials. This is largely due to the differences in the purpose of and resources available for data collection. In clinical trials, data collection involves a limited and pre-specified number of participants (based on a sample size determination). Data collection occurs at pre-specified time intervals (i.e., study visits) for a defined period of time. In addition, the trial data collection system is closed at the end of the study. In contrast, registries are designed to support real time care, quality improvement, and knowledge development. They involve data collection as part of routine care and must embed the process of data collection into the clinical workflow. The data reflect actual practice and patient care. Challenges in this setting may include data collection at every patient visit over an extended period of time, unstandardized visit schedules, and large numbers of data elements needed to support chronic care activities such as population management and pre-visit planning for an entire patient population.

In addition, care centers participating in multi-purpose registries participate voluntarily. Many members of the clinical care team are involved, and resources for data capture and cleaning, such as clinical auditing and source document verification, are substantially less compared with clinical trials. The same staff responsible for transcribing data from the medical record and entering into the electronic case report forms may also be responsible for completing source document verification, in addition to other administrative and/or clinical responsibilities.

Such systems cannot support the data cleaning efforts typical of clinical trials that involve large numbers of queries sent to care centers for response. A key challenge to using data from registries for research is that the quality may not match that of data collected using other, more rigorous and expensive, study support. To date, studies of data quality in registries have focused on retrospective assessments of the “fit for use” model which indicates that the data quality is appropriate for the intended use.

Multi-center registries have used quality improvement methodology to improve patient care and outcomes. These same methods may be extended to interventions that enable teams to improve data quality. As such, we chose to evaluate the impact of a data quality improvement project based on good clinical data management practices within a multi-center registry for clinical care, QI, and research.

Methods

Settings and centers

The ImproveCareNow (ICN) Network is a multi-center international research and quality improvement network whose purpose is to transform the health, care, and costs for all children and adolescents with inflammatory bowel disease (IBD), specifically Crohn’s disease and ulcerative colitis. The network seeks to enable patients, families, clinicians, and researchers to work together to accelerate innovation, discovery, and the application of new knowledge. All 74 participating care centers entering data in the ICN registry from June 2010 through June 2016 were included in this study, representing data from 162,626 visits from 24,309 patients.

The design of the network has been described in detail previously. Briefly, ICN care centers include a mix of large and small academic medical centers and private practices over diverse geographic regions (urban/rural) and include approximately 60 percent of all pediatric gastroenterologists in the United States. Providers at each center receive instruction and ongoing coaching in QI methods, use of tools, and performance reports. Monthly webinars and semi-annual community conferences are held to provide ongoing training. Participating clinicians have developed care guidelines, tools, and processes to reduce variation in care. Centers within the network collect standardized data elements at the time patients are enrolled into the registry, at all follow-up visits, in the event of a hospitalization, and when the patient discontinues participation in the registry. These elements include patient demographics, specific disease characteristics, level of disease activity, test results, treatments, and clinical outcomes. Registry data are used to support chronic care management reports that enable pre-visit planning, population management and patient-tracking, and comparative performance measurement (monthly charts displaying clinical, process, and outcome measures). Registry data can also be used to conduct various types of research, including comparative effectiveness studies. Centers provide their own resources to support data capture as part of their participation in the network.

Development of the quality improvement intervention to enhance data quality

The ICN Data Management Committee was formed as part of the network’s data coordinating center at Cincinnati Children’s Hospital Medical Center to inform the design, development, and testing of the data quality QI process. Representatives include staff from the data coordinating center for the network, as well as individuals from participating ICN clinical care centers. Clinical medicine, research coordination, data management, biostatistics, clinical epidemiology, project management, and QI expertise were all included on the committee. The data coordinating center of the network reports to the network’s executive leadership and is an ongoing network function. The DCC and the committee are accountable for the ongoing maintenance and improvement of data quality. Its work is funded as part of network operations.

A structured QI process was used to design a data quality system with the following aim:

To assist ICN care centers in creating a reliable, accurate, and consistent data collection system and full population registry (all patients, all visits, all data items) so patients can rely on the recommendations generated from the registry, and physicians and researchers can use data to make decisions with confidence about the care and outcomes of children with IBD.

A logic model known as a key driver diagram was developed to identify changes and interventions that support the drivers of good data quality (Figure 1). These key drivers included having consistent and timely information about the population, having reliable data collection practices, and the application of QI methodology to improve data quality. As illustrated in the key driver diagram, specific interventions in several processes were identified: creating and maintaining a full population registry, developing a reliable process for data collection, and monitoring data quality.

Figure 1. ImproveCareNow’s data quality key driver diagram

The data quality improvement effort was built upon ongoing QI training and support processes that were already established for centers. The team initially defined nine measures that measured completeness, consistency, and timeliness of data entered into the ICN registry (see Appendix A). These measures were implemented in April 2011, and the consistency measures were bundled together in late 2013 due to high performance on each measure and similarity of content. The measures still covered the same broad categories (completeness, consistency, and timeliness). Two additional measures for monitoring hospitalization data were added in 2014. Only current measures are presented. The original data quality reports were developed to look like the reports for the clinical process and outcomes measures already utilized by centers and distributed on a monthly basis. By July 2013, the data quality charts were made available electronically in conjunction with the other QI tools in the system, including clinical measures, population management reports, and pre-visit planning reports. In addition, the reporting system was modified so that the results were updated on a nightly basis to align with daily data collection updates. This enabled centers to view the charts at any time, whereas previously changes were not reflected until the next reporting month.

Notes

This presentation is faithful to the original, with only a few minor changes to presentation. Grammar was cleaned up for smoother reading. In some cases important information was missing from the references, and that information was added.