Journal:Quality management system implementation in human and animal laboratories
|Full article title||Quality management system implementation in human and animal laboratories|
|Author(s)||Kachuwaire, Obert; Zakaryan, Arsen; Manjengwa, Julius; Davtyan, Zaruhi; Châtard, Jerome; Orelle, Arnaud; Tumanyan, Pertch; Petikyan, Aida; Hambardzumyan, Nune; Pierson, Antoine|
|Author affiliation(s)||Integrated Quality Laboratory Services, Republican Veterinary-Sanitary and Phytosanitary Laboratory Services Center, Armenian Centers for Disease Control and Prevention|
|Primary contact||Email: kachuwaire at iqls dot net|
|Volume and issue||13|
|Distribution license||Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International|
Background: The ability to rapidly detect emerging and re-emerging threats relies on a strong network of laboratories providing high-quality testing services. Improving laboratory quality management systems (QMS) to ensure that these laboratories effectively play their critical role using a tailored stepwise approach can assist them to comply with World Health Organization's (WHO) International Health Regulations (IHRs), as well as the World Organization for Animal Health's (OIE) guidelines.
Methods: Fifteen (15) laboratories in Armenia's human and veterinary laboratory networks were enrolled into a QMS strengthening program from 2017 to 2020. Training was provided for key staff, resulting in an implementation plan developed to address gaps. Routine mentorship visits were conducted. Audits were undertaken at baseline and post-implementation using standardized checklists to assess laboratory improvements.
Results: Baseline audit general indicator scores ranged from 21% to 46% for human laboratories and 37% to 60% for the veterinary laboratories. Following implementation scores improved, ranging from 7% to 39% for human laboratories and 12% to 19% for veterinary laboratories.
Conclusion: In general, there has been improvement for both human and veterinary laboratories in the areas of QMS implementation, particularly in organizational structure, human resources, equipment management, supply chain management, and data management. Central facilities developed systems that are ready for international accreditation. This One Health strengthening project ensured simultaneous strengthening of both human and veterinary laboratories, which is not a common approach.
Keywords: laboratory quality management, One Health, laboratory assessments, veterinary laboratory, public health laboratory, standardization
A core component within global health security initiatives, including the International Health Regulations (IHRs) and Global Health Security Agenda (GHSA), is the need for responsive and technically competent laboratories. These laboratories play a frontline role in disease detection, surveillance, and response efforts, especially crucial in light of threats from emerging and re-emerging infections of pandemic potential, like the current COVID-19 response. Laboratory results and data generated from these entities are useful if they are reliable and reproducible, eliciting trust and confidence in end users. However, in many low- and middle-income countries (LMICs), laboratory quality standards are hampered by a myriad of factors, including lack of regulations, scarce resources and expertise to set up such systems, and the high cost of international accreditation programs.
Laboratory strengthening efforts that incorporate stepwise implementation of quality management systems (QMS) have been promoted globally since the WHO 2008 Lyon meeting on quality, and the subsequent key global calls to action. An adequate nationwide laboratory system that is able to reliably support outbreak and surveillance activities consists of human health laboratories, among other sectors, including animal, food, water, and environmental health (depending on the context), as most of the emerging and re-emerging disease threats are proving to be zoonotic. Therefore, a QMS implementation strategy that seeks to not just enable human health laboratories but also leverage a One Health approach—where "multiple sectors communicate and work together to achieve better public health outcomes"—is encouraged.
Quality assured diagnostics for both human and veterinary services are key in enhancing efficiencies in the government of Armenia’s (GoA) laboratory testing capacities to detect select agents at a minimal number of safe and secure facilities, as well as in enhancing safe, secure, and sustainable infectious disease surveillance and reporting. In 2017, the GoA and the United States of America Defense Threat Reduction Agency (DTRA) collaborated in the strengthening of human and animal laboratories. They fall under the National Center for Disease Control (NCDCP) for human health and the Food Safety Inspectorate (FSI) for animal health. Through the International Science and Technology Center, Astana Kazakhstan, with Integrated Quality Laboratory Services (IQLS)—providing technical assistance—worked with the GoA and DTRA to address key gaps related to the laboratory QMS. Selected laboratories were composed of central facilities located in the capital city and satellite branches in regional locales (locally Marzes).
IQLS conducted laboratory assessments of both the human and animal laboratory networks. These assessments highlighted that a majority of laboratories were challenged in their QMS. In this article, we present three years of laboratory level QMS implementation (2017–2020). We describe a QMS strengthening approach that uses evidence-based results from laboratory system and on-site laboratory facility assessments to guide implementation, including training and on-site mentorship. We used adapted international tools, allowing for a phased approach as recommended for such settings.
Laboratory quality strengthening process
The baseline state of a QMS was determined through an inception period, consisting of system and site assessment of nine laboratories (human [n = 6] and animal health [n = 3]) in the fourth quarter of 2017 and early 2018. Information from these assessments was subsequently used for further site selection. Pursuant the inception period, fifteen (15) key laboratories that would form the backbone of the laboratory system were purposively selected by a joint GoA/DTRA working group and were composed of the following (see Figure 1):
- seven human health laboratories: Central Reference level (Yerevan) and its six Marz level branches
- two infectious disease hospital laboratories
- six Veterinary laboratories: Central Reference level and five Marz level laboratories
Two training sessions were organized (Level One and Two) to pass along knowledge and skills on QMS implementation. The Level One workshop was designed as a two-week train-the-trainer model to assist laboratory quality managers and their deputies to develop competencies that are transferrable to other staff, focusing on QMS implementation at their laboratories. A teach-back approach was used for the second week to develop knowledge sharing skills among the participants. The Level Two workshop was conducted over one week, bringing together human and veterinary specialists for experience sharing. A local IQLS team composed of an international laboratory specialist and a national laboratory specialist provided direct on-site mentorship alone or in conjunction with visiting international experts, focusing on either veterinary or human health. IQLS teams worked in conjunction with the national quality focal persons on these visits. Onsite visit activities included review of previously developed action plans for furthering progress with laboratory management and quality-focused personnel. Each laboratory was encouraged to form a quality team that would meet weekly to deliberate on QMS activities. Further follow-ups utilized an adapted checklist from the World Health Organization's (WHO) Laboratory Quality Stepwise Improvement process (LQSI), customized by IQLS into a user-friendly Microsoft Excel workbook and validated in other countries. This tool is based on the ISO 15189:2012 Medical laboratories — Requirements for quality and competence standard, broken down into four implementation phases, enabling laboratories to move towards accreditation in a step-by-step manner. Those phases are:
- Phase 1: Ensure that the primary process of the laboratory operates correctly and safely.
- Phase 2: Control and assure quality in a traceable fashion.
- Phase 3: Ensure proper management, leadership, and organization.
- Phase 4: Create continuous improvement and prepare for accreditation.
This approach was chosen by the NCDCP top management, as their Central facility laboratory was targeting accreditation for several analytes. A practical tool developed by IQLS to assess veterinary laboratories (animal health and/or food safety) previously used in Mali and Mauritania (2016) and Pakistan (2017–2018) was selected for the animal sector and customized for Armenia. These tools assisted facilities with tracking implementation of activities. Additional activities included supporting facilities with development and review of standard operating procedures (SOPs) and guidelines, as well as their adaptation and adoption at lower-level facilities.
Assessment process, data capture, and analysis
Both baseline and final assessments for the human and veterinary laboratory networks were conducted by IQLS laboratory specialists. Each laboratory was assessed by three individuals composed of local IQLS team members and international laboratory specialists. Human health laboratory specialists assessed only the human laboratories, whilst veterinary laboratory specialists focused on the animal laboratory network. National quality and biosafety managers accompanied the assessors for skills transfer in assessing the Marz facilities. The assessment process included an initial opening meeting with laboratory top management. This was followed by a walkthrough visit of laboratory facilities, following the sample path from collection to results reporting. Assessors looked for evidence of implementation of quality practices in the different areas visited, including availability of documentation in terms of SOPs, guidelines, and forms or records where applicable. After the laboratory visit, the assessment tool was completed, in conjunction with main stakeholders (i.e., the quality manager or officer, if any; laboratory management; and other lead staff) ensuring that data was verified and reconciled, with additional document verification for availability of corroborative evidence. Finally, feedback was given to the laboratory through a closing meeting with the head of the facility and laboratory management.
Human laboratories were assessed using the adapted WHO-LQSI checklist. All four phases were assessed at the Central Facility, encompassing the complete 12 quality system essentials (QSEs). Nevertheless, this tool was not appropriate for other facilities (i.e., NCDCP branches and hospitals) due to its strict demands, as it assumes in-depth knowledge of QMS and its specific vocabulary. We therefore only assessed phases one and two (meaning basic but robust QMS in place) for 10 QSEs (excluding QSE 11 and 12). The veterinary laboratories tool assesses different aspects of the laboratory with 14 elements. Modules of both tools are outlined in Table 1.
All tools (provided as supplementary documents) are Microsoft Excel files with similar functionalities:
- The tools are composed of different tabs corresponding to the different aspects/quality system essential assessed and include easy export features for data aggregation.
- All tools/files are multilingual, including English and Armenian.
- The tools only include closed questions with drop down list answers, automatically generating a score in percentages. For each question, a specific field allows for comments and information.
- A summary tab displays a comprehensive overview of the assessment.
Descriptive statistics were used to classify laboratory implementation status using the general indicator score (which is the average of all module indicator scores).
The judgement criteria of the performances of the laboratories and modules were as follows: excellent (>90%); very good (>70% but ≤90%); good (>60% but ≤70%); fair (≥50% but ≤60%); weak (≥35% but <50%); very weak (<35%). The same tools were used for baseline and final assessments.
Training and mentorship
In total, 54 laboratory staff were trained in quality management systems.
For the QMS Level One Human laboratories (train-the-trainer), 22 laboratory quality managers and microbiologists were trained. For the QMS Level One Veterinary (train-the-trainer), 12 veterinarians were trained. Twenty-one (21) specialists were trained for the QMS Level 2, composed of 13 human health and nine veterinary health specialists. Trainings exhibited improved scores from pre- and post-test results, with all three sessions recording positive gains (+21%, +15%, +3.33% for QMS Level One human laboratory specialists, QMS Level One veterinary laboratory specialists, and QMS Level Two respectively). Following the training, each participant conducted at least one step-down training for staff at their laboratories, which was verified during mentorship visits.
Assessment results were available for nine human health laboratories (n = 6 with baseline and final results, while n = 3 had final assessment results only as enrolled post-inception). Of the laboratories with both results, the central laboratory (Human 1) had a general indicator score of 46% across all four phases (see Figure 2). However, as previously stated, the rest of the laboratories were not suitable to assess all four levels and were graded based on the two phases. All of the five Marz laboratories with both results scored below 50%, meaning they classified weak to very weak (range 21% to 47%). After implementation, the central facility had a very good average score (73%), whilst the Marz laboratories graded as follows: one had very good score (74%); three had good scores (60%, 65%, and 66%, respectively); one had a fair score (54%); two had weak scores (40% and 47% respectively); and one scored very weak at 31%. (see Figure 2).
Average scores of the different module indicators for baseline assessment showed that out of the 12 QSEs, only two scored above the weak grading: facility and safety 56% (with low scores for facility and safety assessments, biosafety manual and information, infection prevention, and risk group indicators) and non-conforming events 93%, also the highest (Table 2). The lowest scoring QSE was continual improvement (0%), which also remained a weak area even after final assessment (one of the last steps prior to formal accreditation). Following implementation, all QSEs scored fair and above. The highest scoring QSE was documents and records 89% (63% increase) and the lowest was customers (50%) (low scores in service manual and management indicator). Improvement difference in QSE scores from baseline to final assessment ranged from 0% to 63%, with the highest improved score recorded for documents and records.
Three animal health laboratories were assessed during the baseline assessment. The Central facility (Vet 1) scored highest with a grading of good, whilst the other two laboratories had weak grading scores of 37% and 49%, respectively (Figure 3). Following implementation, at final assessment, all laboratories except for one scored fair and above, with the highest score for the Central facility, which scored excellent at 79%. The highest Marz veterinary laboratory had a good score at 61%, with the lowest scoring 46%.
As shown in Table 3, the General indicator average score for the veterinary laboratories improved 8% from weak to fair, mainly due to improvements in analytical quality management (32%), equipment management and supply (16%), and data management (21%). The highest score was obtained for sampling and sampling transportation (75%) and diagnostic capacity (83%). The lowest scoring was forbudget and finances (44%), training and supervision (37%), information technologies (38%), and communication (42%). Quality of installations were the only indicator that regressed, decreasing by 7%.
Implementing QMSs in laboratories is expected to enhance diagnostic testing processes, in turn ensuring high levels of competence for reproducible and accurate results. IQLS implemented a One Health laboratory QMS strengthening project in Armenia focusing on both human and veterinary laboratories, which engaged high level officers through an intersectoral approach, with the Ministries of Health and Agriculture facilitating collaboration. This is not always easy to achieve due to the vertical nature of how these sectors operate. A comparative analysis of the final assessment against baseline scores shows increased scores per the different laboratory QSEs.
Improvement for both networks were in areas of organizational structure, human resources, equipment management, supply chain, and data management. These advances were due to practical implementation of training and the numerous SOPs, procedures, and guidelines that were developed and adopted by all laboratories. Central facilities implemented a QMS to the level of considering accreditation for key analytes. Central facilities showed stronger results since they are reference laboratories linked to international networks, receiving prior support from other entities that raised awareness of QMS concepts early on.
For the human Marz laboratories, all except four laboratories moved from weak to good and or very good. Of the remaining four, one of the laboratories moved from weak to fair, mainly due to staff motivation challenges emanating from employment uncertainties for lead staff. The other three laboratories that remained in the weak zone lacked appointed quality managers for a large part of the implementation period. Two of the lowest results (average 39%) were from the hospital network ,which was not part of the NCDCP network and thus had little to no prior exposure to documentation of QMS activities, in addition to lacking key leading staff emphasizing the need for a system wide focus to lab strengthening during implementation.
Per the QSEs' general indicator scores, there was an average 19% increase for all human laboratories. The non-conforming events QSE showed excellent results. The lowest scoring QSE was on continual improvement, which can be explained by a new QMS system still focused on developing many documents, with limited implementation, especially at Marz level. Personnel, customers, and assessments had fair scores. Personnel nonconformities were a result of gaps including lack of staff appraisal systems, job descriptions, and continuous professional development, which are outside the immediate purview of laboratory operations, requiring long intervention time to change. Customer satisfaction was also an area in progress, with the laboratory user manual—which constituted a major requirement—still in draft format. From the results, the laboratories scored above average on all the modules except continual improvement and customers, not currently applicable for these laboratories recently implementing QMS, thus focusing mostly on document development and process management.
For the veterinary network, the central facility results showed substantial progress during the last two years, evidenced by the general indicator percentage difference of 19%. Of note is the improvement in the critical areas such as analytical quality management, equipment management and supplies, data management, and AST performance. However, diagnostic capacity (testing of bacterial, viral, and parasitic infections) requires more focus to increase functionality of the central facility for disease surveillance and testing of bacterial and viral infections by PCR and ELISA.
The Marz veterinary laboratories, also recorded some improvement (9%). The low level of progress is due to a lack of a dedicated quality and biosafety manager, as FSI have not appointed this person for the veterinary laboratory network due to needing some policy changes from top management. Quality of installations regressed due to limited funding to laboratories, as well as lack of general maintenance that could be only provided by qualified engineers within the network. Areas of progress related to analytical quality management and data management. Other areas showed limited improvement, indicating further attention is required as well as supervision from central level.
There were limitations on generalizing progress, as not all laboratories had baseline results due to a limited number of facilities being assessed during the inception period. However, considering the state of the rest of the Marz level facilities, there is room to conclude improvements were brought about through the implementation of this project. The LQSI tool used to assess human laboratories was appropriate for NCDCP (targeting accreditation) but not for branches. It would have been easier and more result-driven to use an adapted tool, which would have provided much more concrete ideas for improvement than LQSI, which asks for significant knowledge in QMS even for lower-level laboratories that may never be targeted for international accreditation. However, we were able to use the same tool to guide implementation with laboratories following expected outputs from each phase. Similarly, the veterinary laboratory assessment tool was appropriate for the central facility, but not for small rural laboratories, which only perform a limited number of tests.
The improvements demonstrated within the project can be credited to collaboration between local and international specialists working to provide direct mentorship to facilities. Other QMS projects also report the usefulness of mentorship on improving laboratory QMS processes. Mentorship provided a platform for routine interaction with Marz facilities, enabling the sharing of documentation as well as assisting with other implementation challenges, in turn enabling the laboratories to move forward. The mentorship visits also highlighted the need for better communication within the network and continuation of supervision programs post-project implementation (especially where central level staff were aware of availability of policies that were not known at Marz level).
Laboratories showing poor results post-implementation lacked quality managers for long periods of time, demonstrating the importance of a full-time or substantive quality manager. Without a quality manager, progress is slow. The combined Level Two QMS training with veterinary and human specialists was useful, with staff from both sides learning about and from each other's implementation processes and means of solving challenges. Additional activities which were conducted in a One Health aspect included a joint external quality assessment program, as well as the purchase od equipment for both networks to enhance quality testing.
Comparative analysis of final assessment results against baseline scores showed improvement in Armenian laboratory operations. A One Health approach to laboratory strengthening can be implemented with multisectoral stakeholder involvement.
The authors are thankful to the leaders of Armenian MoH, Food Safety Inspectorate, and all participating laboratories for their collaboration and helping to implement project activities.
The original manuscript was developed and drafted by O.K., A.O., J.M., and Z.D, and A.P. provided considerable input into structure and reviewed the manuscript. J.C, P.T, A.P, and N.H reviewed the manuscript.
The development of this publication was supported by the International Science and Technology Center and the United States Defense Threat Reduction Agency project implemented in Armenia under the grant/agreement HDTRA117F0048P00003 Laboratory Strengthening Project in Armenia.
Conflicts of interest
The authors declare no conflict of interest with respect to the research, authorship, and/or publication of this article.
- Wilson, Michael L; Fleming, Kenneth A; Kuti, Modupe A; Looi, Lai Meng; Lago, Nestor; Ru, Kun (1 May 2018). "Access to pathology and laboratory medicine services: a crucial gap" (in en). The Lancet 391 (10133): 1927–1938. doi:10.1016/S0140-6736(18)30458-6. https://linkinghub.elsevier.com/retrieve/pii/S0140673618304586.
- Balajee, S. Arunmozhi; Arthur, Ray; Mounts, Anthony W. (1 December 2016). "Global Health Security: Building Capacities for Early Event Detection, Epidemiologic Workforce, and Laboratory Response" (in en). Health Security 14 (6): 424–432. doi:10.1089/hs.2015.0062. ISSN 2326-5094. http://www.liebertpub.com/doi/10.1089/hs.2015.0062.
- Global Health Security Agenda. "Laboratory Systems". Global Health Security Agenda. https://ghsagenda.org/laboratory-systems/. Retrieved 01 December 2020.
- World Health Organization, ed. (2016). International health regulations (2005) (Third edition ed.). Geneva, Switzerland: World Health Organization. ISBN 978-92-4-158049-6.
- Dzau, Victor; Fuster, Valentin; Frazer, Jendayi; Snair, Megan (28 September 2017). "Investing in Global Health for Our Future" (in en). New England Journal of Medicine 377 (13): 1292–1296. doi:10.1056/NEJMsr1707974. ISSN 0028-4793. http://www.nejm.org/doi/10.1056/NEJMsr1707974.
- Flatland, Bente (1 June 2012). "Veterinary laboratory quality management - it takes a village" (in en). Veterinary Clinical Pathology 41 (2): 171–173. doi:10.1111/j.1939-165X.2012.00442.x. https://onlinelibrary.wiley.com/doi/10.1111/j.1939-165X.2012.00442.x.
- World Health Organization (April 2008). "Joint WHO – CDC Conference on Health Laboratory Quality Systems" (PDF). World Health Organization. https://www.who.int/csr/ihr/lyon/report20080409.pdf.
- Yang, Ji-Rong; Teng, Hwa-Jen; Liu, Ming-Tsan; Li, Shu-Ying (1 April 2017). "Taiwan's Public Health National Laboratory System: Success in Influenza Diagnosis and Surveillance" (in en). Health Security 15 (2): 154–164. doi:10.1089/hs.2016.0104. ISSN 2326-5094. PMC PMC5404250. PMID 28418742. http://www.liebertpub.com/doi/10.1089/hs.2016.0104.
- Fitzmaurice, Arthur G.; Mahar, Michael; Moriarty, Leah F.; Bartee, Maureen; Hirai, Mitsuaki; Li, Wenshu; Gerber, A. Russell; Tappero, Jordan W. et al. (1 December 2017). "Contributions of the US Centers for Disease Control and Prevention in Implementing the Global Health Security Agenda in 17 Partner Countries". Emerging Infectious Diseases 23 (13). doi:10.3201/eid2313.170898. ISSN 1080-6040. PMC PMC5711326. PMID 29155676. http://wwwnc.cdc.gov/eid/article/23/13/17-0898_article.htm.
- Zheng, Zhe; Lu, Yi; Short, Kirsty R.; Lu, Jiahai (1 December 2019). "One health insights to prevent the next HxNy viral outbreak: learning from the epidemiology of H7N9" (in en). BMC Infectious Diseases 19 (1): 138. doi:10.1186/s12879-019-3752-6. ISSN 1471-2334. PMC PMC6371560. PMID 30744562. https://bmcinfectdis.biomedcentral.com/articles/10.1186/s12879-019-3752-6.
- "One Health". World Health Organization. 21 September 2017. https://www.who.int/news-room/questions-and-answers/item/one-health.
- Centers for Disease Control and Prevention. "HHS and USDA Select Agents and Toxins - 7CFR Part 331, 9 CFR Part 121, and 42 CFR Part 73". Federal Select Agent Program. Centers for Disease Control and Prevention. https://www.selectagents.gov/sat/list.htm. Retrieved 17 June 2020.
- World Health Organization. "Laboratory Quality Stepwise Implementation tool". World Health Organization. https://extranet.who.int/lqsi/content/homepage. Retrieved 01 December 2020.
- Okello, Anna L.; Bardosh, Kevin; Smith, James; Welburn, Susan C. (22 May 2014). "One Health: Past Successes and Future Challenges in Three African Contexts" (in en). PLOS Neglected Tropical Diseases 8 (5): e2884. doi:10.1371/journal.pntd.0002884. ISSN 1935-2735. PMC PMC4031173. PMID 24851901. https://journals.plos.org/plosntds/article?id=10.1371/journal.pntd.0002884.
- Yasobant, Sandul; Bruchhausen, Walter; Saxena, Deepak; Falkenberg, Timo (1 December 2019). "One health collaboration for a resilient health system in India: Learnings from global initiatives" (in en). One Health 8: 100096. doi:10.1016/j.onehlt.2019.100096. PMC PMC6606562. PMID 31304229. https://linkinghub.elsevier.com/retrieve/pii/S2352771419300011.
- Polansky, Lauren; Chester, Stephanie; Warren, Melissa; Aden, Tricia; Kennedy, Pamela; Spivey-Blackford, Stacey; Moen, Ann (1 December 2019). "Can mentorship improve laboratory quality? A case study from influenza diagnostic laboratories in Southeast Europe" (in en). BMC Health Services Research 19 (1): 49. doi:10.1186/s12913-018-3840-0. ISSN 1472-6963. PMC PMC6339419. PMID 30658627. https://bmchealthservres.biomedcentral.com/articles/10.1186/s12913-018-3840-0.
- Perrone, Lucy A; Voeurng, Vireak; Sek, Sophat; Song, Sophanna; Vong, Nora; Tous, Chansamrach; Flandin, Jean-Frederic; Confer, Deborah et al. (1 October 2016). "Implementation research: a mentoring programme to improve laboratory quality in Cambodia". Bulletin of the World Health Organization 94 (10): 743–751. doi:10.2471/BLT.15.163824. ISSN 0042-9686. PMC PMC5043202. PMID 27843164. http://www.who.int/entity/bulletin/volumes/94/10/15-163824.pdf.
This presentation is faithful to the original, with only a few minor changes to presentation, spelling, and grammar. We also added PMCID and DOI when they were missing from the original reference. A citation related to One Health (WHO 2017) was added to the introduction as the concept of what One Health is is never formally introduced. No other modifications were made in accordance with the "no derivatives" portion of the distribution license.