Journal:Practical considerations for laboratories: Implementing a holistic quality management system

From LIMSWiki
Jump to navigationJump to search
Full article title Practical considerations for laboratories: Implementing a holistic quality management system
Journal Frontiers in Bioengineering and Biotechnology
Author(s) Pillai, Segaran; Calvery, Jennifer; Fox, Elizabeth
Author affiliation(s) Food and Drug Administration, Booz Allen Hamilton
Primary contact Email: Segaran dot Pillai at fda dot hhs dot gov
Editors Quemada, H.
Year published 2022
Volume and issue 10
Article # 1040103
DOI 10.3389/fbioe.2022.1040103
ISSN 2296-4185
Distribution license Creative Commons Attribution 4.0 International
Website https://www.frontiersin.org/articles/10.3389/fbioe.2022.1040103/full
Download https://www.frontiersin.org/articles/10.3389/fbioe.2022.1040103/pdf (PDF)

Abstract

A quality management system (QMS) is an essential element for the effective operation of research, clinical, testing, or production/manufacturing laboratories. As technology continues to rapidly advance and new challenges arise, laboratories worldwide have responded with innovation and process changes to meet the continued demand. It is critical for laboratories to maintain a robust QMS that accommodates laboratory activities (e.g., basic and applied research; regulatory, clinical, or proficiency testing), records management, and a path for continuous improvement to ensure that results and data are reliable, accurate, timely, and reproducible. A robust, suitable QMS provides a framework to address gaps and risks throughout the laboratory's workflow that could potentially lead to a critical error, thus compromising the integrity and credibility of the institution. While there are many QMS frameworks (e.g., a model such as a consensus standard, guideline, or regulation) that may apply to laboratories, ensuring that the appropriate framework is adopted based on the type of work performed and that key implementation steps are taken is important for the long-term success of the QMS and for the advancement of science. Ultimately, a well-considered QMS ensures accurate results, efficient operations, and increased credibility, enabling protection of public health and safety. Herein, we explore QMS framework options for each identified laboratory category and discuss prerequisite considerations for implementation. An analysis of various frameworks’ principles and conformity requirements demonstrates the extent to which they address basic components of effective laboratory operations and guides optimal implementation to yield a holistic, sustainable framework that addresses the laboratory’s needs and the type of work being performed.

Keywords: quality, quality management system, laboratory, implementation, framework, total quality, reproducibility

Introduction

In order for laboratories to support the public health mission and better address emerging public health challenges, the need for a quality management system (QMS) that facilitates risk-based thinking and enhances assurance of data quality becomes more critical. With the worldwide shock of the coronavirus disease 2019 (COVID-19) outbreak, organizations have been forced to pivot and innovate to address new issues. The rapid response across industries towards COVID-19 and other crises illustrates business resilience and emphasizes the importance of quality management in the laboratory and its organizational culture. However, incidents stemming from gaps in quality management and affecting public health continue to occur. Laboratory errors have a reported frequency of 0.012%–0.6% of all test results, which in turn has huge impact on diagnosis, as 60 to 70% of all diagnoses are made based upon laboratory tests.[1] In addition, there is evidence of an observed data reproducibility crisis in the research community that would benefit from attention and improvement. A recent Nature survey of 1,576 researchers found that 52% of those surveyed agree that there is a crisis of reproducibility; the same survey found that over 70% have failed to reproduce another scientist’s data.[2]

A QMS has numerous benefits that contribute to managing risks in the laboratory, including errors, and mitigating reproducibility crisis concerns. One major benefit is enabling the laboratory to better identify, assess, and address risks faced in the laboratory at all levels of the organization. Poor data quality is one such risk. Some standards, such as the International Organization for Standardization's (ISO's) ISO 31000, are completely dedicated to risk management[3], while other more holistic quality management standards, guidelines, and regulations—such as ISO/IEC 17025—incorporate the concept of risk-based thinking throughout their respective frameworks.[4] In turn, risk management and other aspects of quality management can be built into all three phases of the workflow path (commonly referred to as the "total testing process") of a laboratory: pre-analytical, analytical, and post-analytical.[5]

The pre-analytic phase of a laboratory’s workflow encompasses the activities that are completed prior to operational testing and research (e.g., sample collection and transport). These activities are performed to prepare and support the laboratory in its operations, research, and services. The analytic phase of a laboratory’s workflow encompasses the activities that are completed within the laboratory during operational testing and research. These activities are performed to directly execute the laboratory’s operations, research, and services (e.g., sample testing, experimental studies). Finally, the post-analytic phase of a laboratory’s workflow encompasses the activities that are completed after operational testing and research (e.g., reporting). These activities assure that processes are conducted in a manner that ensures compliance, accuracy, customer satisfaction, and continual improvement.

Each of these three phases must be addressed when implementing a QMS. While someone may suppose that the highest risk of error would occur in the analytic phase, there is now incontrovertible evidence that the majority of laboratory errors occur in the pre-analytical phase (61.9%–68.2%), which are followed by mistakes in the post-analytical phase (18.5%–23.1%) and analytical phase (13.3%–15%).[6] Within the path of workflow, laboratory processes and procedures (P&P) can be organized into 12 management areas, known as quality system essentials (QSEs). The 12 QSEs—originally found in the HS1 and GP26 guidelines developed by the National Committee for Clinical Laboratory Standards (NCCLS)[7][8], today known as the Clinical and Laboratory Standards Institute (CLSI)—are globally recognized principles that address all aspects of a QMS and cover the entire path of workflow of a laboratory.[9] When all of the laboratory procedures and processes are organized into an understandable and workable structure, the opportunity to ensure that all are appropriately managed is increased.[5] Moreover, as safety is a key QSE[9], laboratories such as those handling pathogens and other biosafety risks have the opportunity to evaluate the effectiveness of their biosafety procedures and implement improvements (e.g., take measures to reduce the risk of contamination). An integrated, robust QMS can help the laboratory cope with uncertainties that naturally occur in all laboratory environments.

In his 1993 book Preventing Chaos in a Crisis: Strategies for Prevention, Control, and Damage Limitation, Patrick Lagadec emphasizes that the response to an emergency cannot be developed unless the institution has prepared to adapt[10]:

...the ability to deal with a crisis situation is largely dependent on the structures that have been developed before chaos arrives. The event can in some ways be considered as an abrupt and brutal audit: at a moment’s notice, everything that was left unprepared becomes a complex problem, and every weakness comes rushing to the forefront.

Given the naturally complex operations of the modern laboratory, minimizing preventable problems and applying lessons learned should be a primary goal.

Defining major laboratory categories

Laboratories are distributed widely across the United States and deal with diverse and unique challenges as a result of differing locations, type of work being performed, and federal, state and local regulations. The role of modern laboratories is quickly evolving as the delivery of health care undergoes drastic changes in the face of unprecedented challenges. With drastic changes comes the need to effectively manage them to ensure continued quality of products and services. It will be critical for a laboratory, belonging to any of the below categories, to reevaluate its alignment with the organization’s strategic direction and quality policy, continually assess risk and impact of changes, carefully define organizational roles and responsibilities, and foster a strong culture of quality and safety.

Testing laboratories (including regulatory laboratories)

Testing laboratories conduct conformance testing to ensure the safety, efficacy, and security of a vast array of materials, including, but not limited to, human and veterinary drugs, biological products, medical devices, food, water, cosmetics, radiation-emitting products, and tobacco products. Testing laboratories are driven by and generate results to support statutory obligations to protect the public and minimize risk associated with such products or materials.

Product development and manufacturing laboratories

Product development and manufacturing laboratories are a subset of testing laboratories that conduct routine quality control analysis, as well as pre-market research and development analysis during design phases, in order to ensure conformity of manufactured products (e.g., test reagents, devices to support analysis, testing or diagnostics) against a set of criteria or attributes selected and agreed upon by the institution. Product development and manufacturing laboratories are driven largely by regulatory requirements, or customer requirements and satisfaction.

Basic and applied research laboratories

Research laboratories can be divided into basic and applied research units. While there is some natural overlap between the two, there are key differences. Basic research is conducted to seek understanding and form answers to scientific questions. According to the Association of American Medical Colleges (AAMC), basic science research—often called fundamental or bench research—provides the foundation of knowledge for the applied science that follows.[11] Applied research laboratories conduct studies that are designed to solve practical problems. While considered separate entities from regulatory laboratories, basic and applied research laboratories conduct their work to support and inform the understanding of science, (such as pathogenesis, side effects, efficacy of medical countermeasures, development and evaluation of technology and devices, etc.) and to support regulatory requirements.

Proficiency testing laboratories

Proficiency testing laboratories assess the performance of individual laboratories responsible for specific tests or measurements generating data for regulatory consideration. Results are used to evaluate laboratories’ continuing performance, an important aspect of laboratory quality management and ensuring data integrity.

Clinical laboratories

Clinical laboratories conduct a variety of testing on clinical specimens to evaluate patients’ health. Their work aids in the diagnosis, treatment, and prevention of disease. The Centers for Medicare and Medicaid Services (CMS) regulates clinical laboratory testing performed on human specimens in the United States through the Clinical Laboratory Improvement Amendments (CLIA).[12]

Recommendation: Identify the correct laboratory quality management system framework for the type of laboratory work conducted

Laboratories should strive to gain a thorough understanding of their current state, goals, and objectives prior to selecting and implementing a QMS framework. Determining the major category of the laboratory will help to narrow the focus of the target state and identify if there is any overlapping of categorization that can be addressed in different ways. Capturing the current needs of the laboratory, customers/stakeholders and their requirements, and applicable rules and regulations for which compliance is required will help the laboratory to understand the current state in order to identify the most suitable standard or framework that will govern their QMS and serve as the benchmark for the target state. Once the laboratory's requirements are understood, an appropriate framework or standard for the QMS can then be selected. The subsequent sections provide an analysis of well-known QMS frameworks that can serve to guide laboratories in selecting the most appropriate model for implementation, beginning with a framework crosswalk analysis.

Crosswalk of the quality system essentials and quality management standard requirements

Table 1 presents a crosswalk analysis of well-known quality management framework documents against the framework of 12 QSEs. These QSEs cover all basic elements of a laboratory's QMS. Each standard was reviewed against the QSE framework, aligning a section (e.g., clause, subclause, subpart) to the applicable QSE in order to estimate the coverage of laboratory quality for the given quality management standard.


Fig1 Pillai FrontBioengBiotech2022 10.jpg

Table 1. Crosswalk of common quality management framework documents. Note that this table displays a non-exhaustive list of example applicable clauses, subclauses, subparts, and requirements.

Analysis of quality management standards by laboratory category

It is critical for a laboratory to consider the type of work conducted and resources available when selecting the most suitable framework. For example, a basic research laboratory conducting studies on plant biology may not benefit from choosing to conform to ISO 17043, a standard that specifies general requirements for proficiency testing laboratories.[13] There are two basic elements for consideration of a framework for suitability: 1) its relevance to the type of work performed by the laboratory, and 2) the sustainability of the framework within the laboratory. Based on analysis of the content, potential intended uses, and approaches of quality management framework documents, Table 2 provides examples of suitable QMS framework documents that could best apply to each major laboratory category, along with benefits and limitations determined based on the crosswalk analysis in Table 1.


Fig2 Pillai FrontBioengBiotech2022 10.jpg

Table 2. Benefits and limitations of select quality management framework documents by major laboratory category. LQMS = laboratory quality management system.

Key takeaways

From our analysis and crosswalk of quality documents, it was observed that only the World Health Organization's (WHO’s) Laboratory Quality Management System: Handbook[5], ISO 15189 Medical laboratories — Requirements for quality and competence standard[14], and CLIA addressed the laboratory’s path of workflow (i.e., total testing process). WHO’s handbook explains this concept and its importance, which is valuable considering that error prevention in each phase helps to ensure the quality of outputs. While not written for laboratories, the ISO 9001 standard for a QMS[15] was included in our analysis due to its broad flexibility and ability to integrate with a laboratory’s institutional processes. One notable finding in the crosswalk is that many of the international quality standards, the Food and Drug Administration (FDA) regulation document “Good Laboratory Practice for Nonclinical Laboratory Studies” (21 CFR Part 58; hereafter referred to as “GLP”)[16], and CLIA do not fully address personnel safety as a component of the "Facilities and Safety" QSE, but rather focus on facility maintenance and contamination control. (This could be due to the fact that other national regulatory bodies address environmental and occupational safety, such as the Occupational Safety and Health Administration [OSHA], Nuclear Regulatory Commission [NRC], and Environmental Protection Agency [EPA].) Additionally, it was observed that purchasing and inventory guidance was also limited when compared to WHO’s laboratory QMS framework. For example, while GLP includes requirements for reagent labeling, GLP does not have a requirement for inspection upon receipt or conducting routine inventory. Another key finding is that the clinical quality documents and GLP do not have as strong of a customer focus as the international quality standards; the focus in these documents is geared towards analytical or study conduct and process management. Establishing procedures on customer service or adhering to the customer service aspects of a suitable ISO standard may be advisable in any environment in order to ensure expectations are continually met. Ultimately, the findings in Tables 1 and 2 demonstrate that a holistic laboratory QMS framework, complemented with guidance from additional quality documents, can benefit many laboratories that aim to address all 12 QSEs.

Recommendation: Conduct a gap analysis of the laboratory's current QMS against the selected framework

One of the first key steps needed prior to implementing a QMS, for any organization, is conducting a gap analysis to assess the current state against the target state. Laboratories of all types can take this approach to leverage gap analysis data gathered to identify areas that need to be improved or prioritized to achieve successful implementation (and certification, if pursued). Once the major category of the laboratory is determined, needs and requirements are understood, and the correct framework is selected by the laboratory (and institution, as applicable), they can begin their gap analysis. The gap analysis involves evaluating their QMS against the selected framework, which serves as the benchmark, to identify where P&P are nonexistent or need improvement. It is recommended that the organization discuss and agree upon the prioritization of gaps that need to be addressed. The next subsection provides guidance as to the recommended prioritization level of QSEs for each major laboratory category.

QSE prioritization for consideration

The 12 QSEs of a laboratory QMS, including their key sub-elements (see Supplementary Table S1), are intended to support general quality management of laboratory operations; as such, implementation can be modified or prioritized based on laboratory work performed. The QSEs can fall into two basic levels of importance depending on the type of laboratory work being performed: beneficial and critical. Beneficial QSEs, while not as prioritized as critical QSEs, are recommended for effective, optimized implementation of a QMS in the laboratory (see Supplementary Table S2 for detail and visualization).

Recommendation: Addressing gaps in standard requirements will help to minimize risk

While there are many benefits to applying each individual quality standard or guideline examined in this article, there are also potential limitations depending on the laboratory category and work performed. For example, ISO/IEC 17025 has less focus on personnel safety when compared to the QMS framework detailed in the WHO's Laboratory Quality Management System: Handbook[5], and the GLP regulations do not address customer satisfaction. Laboratory operations are complex and often have several different sets of requirements they must meet, along with processes and outputs they must monitor at each phase of the workflow in the laboratory. Therefore, applying a holistic, comprehensive system of quality management standards is recommended to incorporate different types of requirements (e.g., customer-specific requirements, clinical outcome criteria) and further reduce the risk of an occurrence or inaccurate, untimely, or unreliable results. Occurrences often take the form of laboratory errors, which can have significant consequences.

Regarding clinical laboratories specifically, the focus on analytical quality for many years has resulted in fewer process deviations and an improved error rate that surpasses most other areas in healthcare.[17] However, given the multiple lines of evidence suggesting that most errors in the total path of workflow actually fall outside of the analytical phase, and that the pre- and post-analytical steps have been found to be much more vulnerable[18], only minor attention is still paid to these areas in many laboratories. According to a recent survey among European laboratories, about a third of the laboratories that monitor pre-analytical errors perform further analysis and, even when a statistical analysis is made, approximately 25% of them remain inactive against unsatisfactory results.[6] Implementing a holistic QMS helps to ensure that proper evaluation and application of corrective/preventive actions are conducted, as they are critical for prevention of recurring errors.

While a laboratory need not pursue accreditation to multiple standards, as this would require additional time and resources, many laboratories can choose the most suitable quality framework (while assuring compliance with applicable regulatory requirements [e.g., CLIA]) and incorporate aspects of others—including the plethora of standards available that were not analyzed in this article—to address gaps such as those identified in our key takeaways. For example, if a testing laboratory maintains an accreditation to ISO/IEC 17025 but would like to direct additional attention towards improving personnel safety training procedures, the laboratory may consider implementing the "Facilities and Safety" QSE[5] as a guideline in addition to the ISO/IEC 17025 standard. In another instance, a high-containment research laboratory that follows GLP may consider adopting elements of ISO 35001, a biorisk management standard.[19] A holistic QMS that addresses gaps can remain sustainable long-term, as it takes into consideration that risks, priorities, and needs are continuously evolving, and different standards and guidelines involved in the system can take priority in different situations; additionally, certain sources of information may be more appropriate for defining different types of criteria.[20]

In regard to research laboratories, for which conforming entirely to a quality standard may be restrictive, a multi-faceted approach can be beneficial. A French agency that implemented a QMS for their research laboratory reported that[21]:

...although the use of a single QA [quality assurance] system has been proposed for both routine and research activities, it is now more and more accepted by the scientific community that specific standards are required or preferred when performing research activities. This opinion is based upon the limitations of the standards dedicated to analytical laboratories; these limitations arise from the nature of research activities and from the requirements of these standards (e.g., rigid organizational structure, pre-defined methods).

Any laboratory that has a holistic QMS in place is prepared with methodology to adapt and address changes, new risks, and additional competencies that may be required to optimize performance.

Discussion

There can be a persistent mindset among laboratorians that quality is about checking boxes, and if the laboratory wins certification to a standard, they’ve checked all the boxes and maintain the status quo. But quality in the laboratory is a multi-faceted system that involves much more than that to ensure continued reliability and repeatability of experiments and tests, accuracy of data, and safety of personnel. As laboratories conduct very different types of work, such as basic research, clinical testing/diagnostics, regulatory testing, etc., there is currently not a one-size-fits-all approach to quality that can be applied to all types of laboratories effectively. Digging deeper into a quality management framework and ascertaining other elements that may be needed (or conversely, aspects that would be extraneous to implement for the laboratory’s purposes) to address the complexity, nuance, and requirements of the laboratory can help to implement and sustain a successful QMS.

The findings from our analysis in Table 1 demonstrate the extent to which quality management framework documents address each of the 12 QSEs in the framework outlined in the WHO's Laboratory Quality Management System: Handbook. It was determined that not all of the QSEs are fully addressed by a single framework document (e.g., GLP does not address the "Continual Improvement" QSE), and thus a gap may exist if implemented alone. In addition, the findings in Table 2 indicate that there are different benefits and potential limitations associated with each individual quality management framework, dependent on the type of laboratory. For example, the WHO's Laboratory Quality Management System: Handbook may be the most suitable framework for a basic or applied research laboratory, as limitations were not identified for this laboratory category, but this framework alone may not be the most ideal for a manufacturing laboratory that needs to be accredited to a quality standard to meet a customer requirement.

From the results in both tables, we can conclude that a holistic QMS framework complemented with guidance from multiple quality documents can benefit many laboratories that aim to address all 12 QSEs. Although a perfect QMS is not feasible and there will always be technical and resource limitations, a holistic and robust QMS that addresses all phases of the path of workflow to mitigate risk of errors is conceivable. A robust QMS requires identification of the benefits, suitability, and sustainability of the effort for the laboratory; a thorough understanding and incorporation of all relevant requirements; and monitoring and evaluation at the pre-analytical, analytical, and post-analytical phases. A QMS that properly integrates all relevant requirements, promotes risk-based thinking, and addresses the entire workflow of a laboratory can ultimately promote the generation of good and relevant data, greater confidence in the quality of data and results, best practices, and a culture of responsibility and safety in the laboratory.

Supplementary material

  • Data Sheet 1.docx: Contains Supplementary Table 1 and its sub-elements of the 12 QSEs, and Supplementary Table 2 and its QSE prioritization matrix for QMS implementation.

Abbreviations, acronyms, and initialisms

  • AAMC: Association of American Medical Colleges
  • CLIA: Clinical Laboratory Improvement Amendments
  • CMS: Centers for Medicare and Medicaid Services
  • COVID-19: coronavirus disease 2019
  • EPA: Environmental Protection Agency
  • FDA: Food and Drug Administration
  • GLP: good laboratory practice
  • ISO: International Organization for Standardization
  • NRC: Nuclear Regulatory Commission
  • P&P: processes and procedures
  • QA: quality assurance
  • QMS: quality management system
  • QSE: quality system essential
  • WHO: World Health Organization

Acknowledgements

The findings and conclusions in this paper are those of the author(s) and do not represent the official position of the Department of Health and Human Services, Food and Drug Administration, or Booz Allen Hamilton.

Author contributions

All authors listed have made a substantial, direct, and intellectual contribution to the work and approved it for publication.

Conflict of interest

Author EF was employed by company Booz Allen Hamilton. The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

References

  1. ↑ Agarwal, Rachna (1 May 2014). "Quality-Improvement Measures as Effective Ways of Preventing Laboratory Errors" (in en). Laboratory Medicine 45 (2): e80–e88. doi:10.1309/LMD0YIFPTOWZONAD. ISSN 0007-5027. https://academic.oup.com/labmed/article-lookup/doi/10.1309/LMD0YIFPTOWZONAD. 
  2. ↑ Baker, Monya (26 May 2016). "1,500 scientists lift the lid on reproducibility" (in en). Nature 533 (7604): 452–454. doi:10.1038/533452a. ISSN 0028-0836. https://www.nature.com/articles/533452a. 
  3. ↑ International Organization for Standardization (February 2018). "ISO 31000:2018 Risk management — Guidelines". International Organization for Standardization. https://www.iso.org/standard/65694.html. 
  4. ↑ International Organization for Standardization (November 2017). "ISO/IEC 17025:2017 General requirements for the competence of testing and calibration laboratories". International Organization for Standardization. https://www.iso.org/standard/66912.html. 
  5. ↑ 5.0 5.1 5.2 5.3 5.4 World Health Organization (2011). "Laboratory Quality Management System: Handbook" (PDF). World Health Organization. ISBN 9789241548274. http://apps.who.int/iris/bitstream/handle/10665/44665/9789241548274_eng.pdf?sequence=1. 
  6. ↑ 6.0 6.1 Mrazek, Cornelia; Lippi, Giuseppe; Keppel, Martin H.; Felder, Thomas K.; Oberkofler, Hannes; Haschke-Becher, Elisabeth; Cadamuro, Janne (15 June 2020). "Errors within the total laboratory testing process, from test selection to medical decision-making - A review of causes, consequences, surveillance and solutions". Biochemia Medica 30 (2): 020502. doi:10.11613/BM.2020.020502. ISSN 1846-7482. PMC 7271754. PMID 32550813. https://pubmed.ncbi.nlm.nih.gov/32550813. 
  7. ↑ National Committee for Clinical Laboratory Standards (November 2004). "CLSI HS01-A2". ANSI. https://webstore.ansi.org/standards/clsi/clsihs01a2. 
  8. ↑ National Committee for Clinical Laboratory Standards (November 2004). "CLSI GP26-A3". ANSI. https://webstore.ansi.org/standards/clsi/clsigp26a3. 
  9. ↑ 9.0 9.1 Clinical and Laboratory Standards Institute (2022). "Quality System Essentials". Clinical and Laboratory Standards Institute. https://clsi.org/standards-development/quality-system-essentials/. 
  10. ↑ Lagadec, Patrick (1993). Preventing chaos in a crisis: strategies for prevention, control, and damage limitation. London ; New York: McGraw-Hill. ISBN 978-0-07-707774-7. 
  11. ↑ Association of American Medical Colleges (2022). "Basic Science". Association of American Medical Colleges. https://www.aamc.org/what-we-do/mission-areas/medical-research/basic-science. 
  12. ↑ "Title 42, Chapter IV, Subchapter G, Part 493". Electronic Code of Federal Regulations. National Archives. https://www.ecfr.gov/current/title-42/chapter-IV/subchapter-G/part-493. Retrieved 02 December 2021. 
  13. ↑ International Organization for Standardization (February 2010). "ISO/IEC 17043:2010 Conformity assessment — General requirements for proficiency testing". International Organization for Standardization. https://www.iso.org/standard/29366.html. 
  14. ↑ International Organization for Standardization (November 2012). "ISO 15189:2012 Medical laboratories — Requirements for quality and competence". International Organization for Standardization. https://www.iso.org/standard/56115.html. 
  15. ↑ International Organization for Standardization (September 2015). "ISO 9001:2015 Quality management systems — Requirements". International Organization for Standardization. https://www.iso.org/standard/62085.html. 
  16. ↑ "21 CFR Part 58 - Good Laboratory Practice for Nonclinical Laboratory Studies" (PDF). Federal Register 43 (247): 60013–20. 22 December 1978. https://archives.federalregister.gov/issue_slice/1978/12/22/59967-60019.pdf. 
  17. ↑ Hawkins, Robert (1 January 2012). "Managing the Pre- and Post-analytical Phases of the Total Testing Process" (in en). Annals of Laboratory Medicine 32 (1): 5–16. doi:10.3343/alm.2012.32.1.5. ISSN 2234-3806. PMC PMC3255486. PMID 22259773. http://annlabmed.org/journal/view.html?doi=10.3343/alm.2012.32.1.5. 
  18. ↑ Lippi, Giuseppe; Becan-McBride, Kathleen; Behúlová, Darina; Bowen, Raffick A.; Church, Stephen; Delanghe, Joris; Grankvist, Kjell; Kitchen, Steve et al. (1 January 2013). "Preanalytical quality improvement: in quality we trust" (in en). Clinical Chemistry and Laboratory Medicine (CCLM) 51 (1): 229–241. doi:10.1515/cclm-2012-0597. ISSN 1437-4331. https://www.degruyter.com/document/doi/10.1515/cclm-2012-0597/html. 
  19. ↑ International Organization for Standardization (November 2019). "ISO 35001:2019 Biorisk management for laboratories and other related organisations". International Organization for Standardization. https://www.iso.org/standard/71293.html. 
  20. ↑ Westgard, J. O. (1 January 1999). "The need for a system of quality standards for modern quality management" (in en). Scandinavian Journal of Clinical and Laboratory Investigation 59 (7): 483–486. doi:10.1080/00365519950185201. ISSN 0036-5513. http://www.tandfonline.com/doi/full/10.1080/00365519950185201. 
  21. ↑ Biré, Ronel; Tufféry, Guy; Lelièvre, Hélène; Dragacci, Sylviane (1 October 2004). "The quality-management system in research implemented in the food and food process quality research laboratory of the French Food Safety Agency" (in en). Accreditation and Quality Assurance 9 (11-12): 711–716. doi:10.1007/s00769-004-0857-3. ISSN 0949-1775. http://link.springer.com/10.1007/s00769-004-0857-3. 

Notes

This presentation is faithful to the original, with only a few minor changes to spelling, grammar, and presentation. The PMCID and DOI were also added when they were missing from the original reference. The original does not provide a citation for the claims concerning the 12 quality system essentials (QSEs); this version adds a citation. The original's singular footnote was turned into a citation for this version. For this version, more historical context of the origin of QSEs was added.

Retrieved from "https://www.limswiki.org/index.php?title=Journal:Practical_considerations_for_laboratories:_Implementing_a_holistic_quality_management_system&oldid=51278"