Journal:Laboratory information management system for COVID-19 non-clinical efficacy trial data
|Full article title||Laboratory information management system for COVID-19 non-clinical efficacy trial data|
|Journal||Laboratory Animal Research|
|Author(s)||Yoon, Suhyeon; Noh, Hyuna; Jin, Heejin; Lee, Sugyoung; Han, Soyul; Kim, Sung-Hee; Kim, Jiseon; Seo, Jung S.; Kim, Jeong J.; Park, In H.; Oh, Jooyeon; Bae, Joon-Yong; Lee, Gee E.; Woo, Sun-Je; Seo, Sun-Min; Kim, Na-Won; Lee, Youn W.; Jang, Hui J.; Hong, Seung-Min; An, Se-Hee; Lyoo, Kwang-Soo; Yeom, Minjoo; Lee, Hanbyeul; Jung, Bud; Yoon, Sun-Woo; Jang, Hung-Ah; Seok, Sang-Hyuk; Lee, Yu J.; Kim, Seo Y.; Kim, Young B.; Hwang, Ji-Yeon; On, Dain; Lim, Soo-Yeon; Kim, Sol P.; Jang, Ji Y.; Lee, Ho; Kim, Kyoungmi; Lee, Hyo-Jung; Kim, Hong B.; Park, Hun W.; Jeong, Dae G.; Song, Daesub; Choi, Kang-Seuk; Lee, Ho-Young; Choi, Yang-Kyu; Choi, Jung-ah; Song, Manki; Park, Man-Seong; Seo, Jun-Yeong; Nam, Ki T.; Shin, Jeon-Soo; Won, Sungho; Yun, Jun-Won; Seong, Je K.|
|Author affiliation(s)||Seoul National University, RexSoft Corp., Yonsei University College of Medicine, Korea University College of Medicine, International Vaccine Institute, Konkuk University, Seoul National University Bundang Hospital, Chonbuk National University, Korea University, Korea Research Institute of Bioscience and Biotechnology, Kangwon National University, National Cancer Center, Dongguk University, Yonsei University College of Medicine,|
|Primary contact||snumouse at snu dot ac dot kr|
|Volume and issue||38|
|Distribution license||Creative Commons Attribution 4.0 International|
Background: As the number of large-scale research studies involving multiple organizations producing data has steadily increased, an integrated system for a common interoperable data format is needed. For example, in response to the coronavirus disease 2019 (COVID-19) pandemic, a number of global efforts are underway to develop vaccines and therapeutics. We are therefore observing an explosion in the proliferation of COVID-19 data, and interoperability is highly requested in multiple institutions participating simultaneously in COVID-19 pandemic research.
Results: In this study, a laboratory information management system (LIMS) has been adopted to systemically manage, via web interface, various COVID-19 non-clinical trial data—including mortality, clinical signs, body weight, body temperature, organ weights, viral titer (viral replication and viral RNA), and multi-organ histopathology—from multiple institutions. The main aim of the implemented system is to integrate, standardize, and organize data collected from laboratories in multiple institutes conducting COVID-19 non-clinical efficacy testing. Six animal Biosafety level 3 (BSL-3) institutions proved the feasibility of our system. Substantial benefits were shown by maximizing collaborative high-quality non-clinical research.
Conclusions: This LIMS platform can be used for current and future outbreaks, leading to accelerated medical product development through the systematic management of extensive data from non-clinical animal studies.
Keywords: SARS-CoV-2, COVID-19, non-clinical, laboratory information management system, data
The novel severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has resulted in the serious respiratory-related coronavirus disease 2019 (COVID-19) pandemic. It was first reported in Wuhan, the capital of Hubei, China, in late December 2019 and has attracted international attention. After four months, it spread to multiple countries, leading to more than one million confirmed cases worldwide. SARS-CoV-2, a single-stranded RNA-enveloped virus, enters the host cells by binding its structural spike (S) protein to cell surface receptors, angiotensin-converting enzyme 2 (ACE2). After entering the host cells, the virus hijacks the cell to undergo viral replication. During this process, an aberrant host immune condition called a cytokine storm can occur in response to SARS-CoV-2 infection, which is characterized by high concentrations of pro-inflammatory cytokines and chemokines, including tumor necrosis factor-α, interleukin-1, and interleukin-6, resulting in severe COVID-19 associated with excessive inflammation.
Along with understanding the mechanism of transmissibility and pathogenesis of SARS-CoV-2, significant efforts are being devoted to developing novel vaccines and therapeutic agents to prevent or treat COVID-19. In particular, various COVID-19-related research has been conducted with the participation of multiple organizations. Among such studies, animal models of monkeys, cats, ferrets, hamsters, and human angiotensin converting enzyme 2 (hACE2) transgenic mice, capable of conducting controlled experiments, have made significant contributions to vaccine development. Because various parameters such as mortality, clinical signs, body weight, organ weights, viral titer (viral replication and viral RNA), and multi-organ histopathology should be comprehensively analyzed for a reliable interpretation of the results, there is an increasing need for an integrated system in which multiple institutions can apply a streamlined analysis and input the results simultaneously when conducting large-scale non-clinical animal efficacy studies (which tend to have different protocols and regulations than clinical studies) of a COVID-19 vaccine.
When experimental data are collected from multiple institutions, “organization-level effects” may occur owing to non-standardization in the data coding process and coding errors, which can act as a batch effect in a data analysis. In large-scale experimental multi-center research, it is necessary to prevent the loss of reliability in the data analysis caused by batch effects; however, this is a difficult goal to achieve without an integrated system. An effective approach to overcoming the limitations caused by the prior-mentioned batch effects is to use a laboratory information management system (LIMS).] A LIMS systematizes the research process by automating data-related tasks that were conducted manually using information technology, and simultaneously realizes various needs, such as the standardization of experimental data, sharing, problem tracking, and the creation of reports based on document templates. The accumulated data stored in a LIMS can be useful in understanding the current clinical knowledge and making decisions, which is a remarkable advantage in research on rapidly changing epidemics such as COVID-19.
Despite the advantages of a LIMS, no systems allowing multiple institutions to participate simultaneously in research related to COVID-19 have yet been proposed among developed open-source LIMSs. Given that, we aimed to clearly understand the standard procedures of non-clinical trials for COVID-19 therapeutics and vaccines and to design and implement a LIMS suitable for these research purposes.
COVID-19 data management system description
The COVID-19-specific LIMS was developed to manage accumulated data for a large-scale multicenter study of actual non-clinical efficacy trials for COVID-19. It provides several distinctive user-friendly features that are not provided in other COVID-19 LIMS (see Table 2, later, for details).
The COVID-19 non-clinical LIMS provides a web-based interface, and its implementation requires both front- and back-end development. The back-end of the COVID-19 LIMS was developed to run on PHP 7.0, MySQL 5.2, and nodeJS 11. Its front-end supports HTML5 and can be run on a modern web browser. The proposed system functionality was built with external libraries such as PHPOffice, Headless Chromium, and Semantic UI.
User management and functionalities
Our COVID-19 non-clinical trial LIMS has four different types of users with different authorizations to handle multiple organizations: a chief director, a local administrator and researcher from each organization, and a pathologist (Fig. 1). The role of the chief director includes new project initiation, modification, deactivation, user account management, and data management. The chief director can also control the registration of new organizations and the removal of existing organizations. The role of the local administrators is the same as that of the chief director, except that they are limited to the organization to which they belong. This enables the efficient management of each organization, as well as data security. Researchers can manage data entry and removal and check the project's progress. Pathologists can conduct pathological analyses, pathological data entry, and related functions. Furthermore, multiple user types for multiple roles are allowed. If multiple authorizations are granted, the screen manual is shown based on the user’s choice.
The proposed system, which applies a convenient user interface, was created to provide many functions related to data management for multi-center research. The detailed function list is presented in Table 1.
Functions can be classified into four different categories: access management, trial, audit, and comparison. The access management function forms the basis of the system and manages the user group. The trial function controls the entire project process, including initiation, progression, and finalization. The audit function records all activities that occur in the system and implements data entry, cancelation, and restoration. The comparison function provides several plots according to the investigational product (new drug/repositioning, vaccine/therapeutic), sponsor, experimental animals (hamster model/hACE2 mouse model), research objective, and study site. It performs access control and visualization by classifying all data accumulated in the system.
Data security and audit
The proposed system provides several functions for data security and audits. For security, an access management system based on the access control list (ACL) was incorporated to protect data entry. Unintended data leakage was minimized by providing limited access points to users according to the user access levels. For each running instance, data security at the terminal level was obtained through secure HTTPS-based communication. For audits, the proposed system records, monitors, and manages all user activities, including data entry, by generating a separate database. Furthermore, it provides a history-tracking function to maintain data integrity. The chief director can identify the path of a data contamination and restore it to its original state, which minimizes the problem related to data entry or damage to the raw data.
Data entry and monitoring are possible for each institution, which makes it possible for the local administrator of each institution to check the data status in real time. It is also composed of a system in which the chief director supervises the overall project progress of each organization in real time. In addition, by systematically managing various types of non-clinical data and enabling collaboration between multiple organizations, researchers are expected to be able to quickly derive the desired results. Figure 2 shows the workflow of the COVID-19 non-clinical trial LIMS for vaccine and therapeutics treatments.
Graphics and analysis functions
The COVID-19 non-clinical trial LIMS provides various visualizations that can review the data characteristics at each stage for each institution, as well as optimized visualizations such as line diagrams, bar diagrams, and survival curves according to the data type for awareness of current situations and early detection of unusual situations. The proposed system generates suitable plots for a correspondence analysis according to the data types. In particular, a comparison function was developed to facilitate mutual comparisons according to user-selected conditions, and the system can also generate easy-to-read study-specific graphs.
Data downloading and result reporting
After obtaining approval from the chief director and administrators of each organization, the stored data can be downloaded as an Excel or Word file. In particular, it is possible to create an integrated Word file that includes a data description table, summary plot, and raw data, and all currently entered data can be downloaded as a multi-sheet Excel file with a single click for convenient data management. In addition, the proposed system can be used to monitor the data entry status of each participating organization and simultaneously create reports with a function for automatically summarizing the analysis results.
In applying important commands such as research initiation and termination, a notification system through a mail and messenger (limited to South Korea) notification service was established for relevant individuals such as chief directors and administrators. When a major change occurs in the research, the manager can quickly identify and respond to any unintended situations.
Data management and recording
Data entry control can be achieved according to the characteristics of each data type, and the user input is automatically controlled according to the condition of each stage of the study. Human error is minimized by limiting the range of unobservable values for each data parameter, and entry omission is prevented by notifying the data that need to be input according to the investigation schedule, current date, and data entry status (Fig. 3). In addition, all data entry activities were recorded, and an audit function was reviewed for each dataset. If necessary, an unsuitable data entry can be canceled to prevent data errors. On the data entry screen, users can check all data during the full research period (Fig. 4A) or partial data by filtering the data based on date (Fig. 4B).
Non-clinical studies are essential for identifying the potential effects of promising drug candidates prior to entering human clinical trials. Numerous animal studies have been conducted in the development of therapeutic agents and vaccines during the recent COVID-19 pandemic, leading to the need for rapid and accurate non-clinical efficacy analyses in standardized animal study design protocols. The inoculation of hACE2 transgenic mice with SARS-CoV-2 induced weight loss, increased the viral titer in the lungs, and prompted histopathological changes with increased cytokine levels and interstitial pneumonia. In addition, infection with SARS-CoV-2 in a golden Syrian hamster model induced signs resembling COVID-19 disease, including decreased activity, weight loss, high lung viral load, and the presence of T-lymphocytes in the respiratory tract. Therefore, systematic management of these extensive data—including mortality, clinical signs, body weight, body temperature, viral titer (viral replication and viral RNA), organ weights, and multi-organ histopathology—that can be observed in COVID-19 research using animal models such as hACE2 transgenic mice and hamsters is required for a reliable evaluation and development of a therapeutic agent or vaccine.
The implemented LIMS has many distinctive features providing many user-friendly functions. First, the implemented system classifies COVID-19 non-clinical trial topics into six sub-themes (investigational product, types of investigational product, sponsor, experimental animals, research objective, and study site). It also applies access control and visualization by classifying the research accumulated in the system. Second, using a streamlined analysis, it creates a four-step process related to research progress, pathology analysis, result entry, and report generation. Access control and customized user interfaces are provided for each step, which highlights the work to be done and provides step-by-step guidance as a project planner. Third, a pathology analysis was incorporated to organize various types of pathology data and support the resulting input from pathologists. Finally, the data are organized systematically, and raw data, summary tables, and graph visualizations are documented. Many of these features were found not to be available in open-source LIMS. Table 2 provides a brief comparison of other existing open-source LIMS and the proposed COVID-19 non-clinical trial LIMS.
The COVID-19 non-clinical trial data management system developed in this study consists of a mode for the chief director, administrators, researchers of each organization, and pathologists. It aims to improve the completeness of the database by managing the data entry process for each institution. To confirm this, we applied the proposed system to an actual COVID-19 non-clinical multi-center study. For the actual application through the proposed system, 32 COVID-19 non-clinical efficacy studies were conducted by six animal Biosafety level 3 (BSL-3) institutions in South Korea from July 2020 to February 2021 via research funding. During the implementation period, the project progress was monitored by the chief director in real time through the management of the data entry process for each organization. In addition, the administrators of each institution were able to check and manage the data entry status of the researchers in real time. Researchers were able to review the status of the data entry by outputting descriptive statistics and graphics, such as histograms or box plots. The administrator of the institution mainly used the function for automatically generating a report based on data entry, which enabled the efficient management and monitoring of the study.
As research on infectious diseases steadily increases after the COVID-19 pandemic, such studies are usually conducted as joint research involving multiple centers. Therefore, a system for efficiently managing multi-center laboratory data is required, and standardization is essential for integrating the units of different studies. The proposed COVID-19 non-clinical trial LIMS, designed for organizing laboratory sample management, analysis, and reporting of non-clinical COVID-19 test results, can be beneficial for researchers in large-scale studies where multiple organizations participate in the production of data. The proposed system enables experimental data management, data sharing, and data analysis through a web-based interface. The system focuses on practical functionality such as the construction of management systems, data quality control processes, various graphics and analysis modules, result reporting functions, and data security and notification functions by systematically implementing access management, test execution, auditing, and comparisons. Large-scale data can therefore be managed more quickly and effectively. In particular, the proposed system is focused on multi-center laboratory data collection and the prevention of various problems owing to the non-standardization of the data coding processes at each institution. Furthermore, it was shown through a streamlined analysis that multi-center non-clinical trials can be systematically conducted.
The contribution of the COVID-19 non-clinical trial LIMS toward vaccine and therapeutics development has focused on providing an environment helping health care professionals and researchers establish new hypotheses to solve various problems in research by investigating trends related to COVID-19 prevention and treatment, as well as the use of collected databases. As such, we anticipate the high adaptability of our system will be a boon in helping overcome COVID-19.
Despite these desirable properties, further challenging issues remain to be resolved. Representatively, advanced statistical analysis function modules should be applied to the system. Because the proposed system does not support analysis modules such as descriptive statistics, data accumulated in the system must be downloaded and processed individually when a statistical analysis is required. To complement this, a comparison function module was developed, and the researchers can compare the data of each study in a graph by selecting the components to be compared. The development of linking the accumulated data in LIMS with a cloud-based statistical analysis program will bring considerable benefits for researchers in the near future.
As research related to COVID-19 will continue to expand and become more complex in the future, the systematic storage and structuring of research data is expected to become more important. In conclusion, the present study provided a new usage for a LIMS, enabling efficient and reliable documentation, management, and reporting of non-clinical COVID-19 test results. This tool will help researchers test therapeutic agents and vaccines against COVID-19, and can be adapted to different research projects. In addition to COVID-19 research, this LIMS platform can also be used for future pandemic-inducing diseases driven by pathogens currently unknown, leading to an accelerated development of medical products through the systematic management of extensive data from non-clinical animal studies.
Abbreviations, acronyms, and initialisms
ACE2: angiotensin-converting enzyme 2
ACL: access control list
BSL-3: Biosafety level 3
COVID-19: coronavirus disease 2019
LIMS: laboratory information management system
SARS-CoV-2: severe acute respiratory syndrome coronavirus 2
Suhyeon Yoon, Hyuna Noh, and Heejin Jin are equal first authors.
SY, HN, and HJ participated in the design of the study, contributed to the analyzing the data and wrote the manuscript. SL and SH contributed to the analyzing the data and provided technical advices. SHK, JK, JSS, JJK, IHP, JO, JYB, GEL, SJW, SMS, NWK, YWL, HJJ, SMH, SHA, KSL, MY, HL, BJ, SWY, JAK, SHS, YJL, SYK, YBK, JYH, DO, SYL, SPK, and JYJ contributed to the analyzing the data. HL, KK, HJL, HBK, JWP, DGJ, DS, KSC, HYL, YKC, JC, MS, MSP, JYS, KTN, and JSS provided technical advices. SW, JWY, and JKS participated in the design of the study, in discussions and reviewed the manuscript. All authors read and approved the final manuscript.
This research was supported by the National research foundation of Korea (NRF) grant funded by the Korea government (MSIT) (2020M3A9I2109027 and 2021M3H9A1030260).
Availability of data and materials
The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.
The authors declare that they have no conflict of interest.
- Khan, N. (8 January 2020). "New Virus Discovered by Chinese Scientists Investigating Pneumonia Outbreak". The Wall Street Journal. https://www.wsj.com/articles/new-virus-discovered-by-chinese-scientists-investigating-pneumonia-outbreak-11578485668. Retrieved 27 May 2022.
- Morens, David M.; Daszak, Peter; Taubenberger, Jeffery K. (2 April 2020). "Escaping Pandora’s Box — Another Novel Coronavirus" (in en). New England Journal of Medicine 382 (14): 1293–1295. doi:10.1056/NEJMp2002106. ISSN 0028-4793. http://www.nejm.org/doi/10.1056/NEJMp2002106.
- Sanders, James M.; Monogue, Marguerite L.; Jodlowski, Tomasz Z.; Cutrell, James B. (13 April 2020). "Pharmacologic Treatments for Coronavirus Disease 2019 (COVID-19): A Review" (in en). JAMA. doi:10.1001/jama.2020.6019. ISSN 0098-7484. https://jamanetwork.com/journals/jama/fullarticle/2764727.
- Hoffmann, Markus; Kleine-Weber, Hannah; Schroeder, Simon; Krüger, Nadine; Herrler, Tanja; Erichsen, Sandra; Schiergens, Tobias S.; Herrler, Georg et al. (16 April 2020). "SARS-CoV-2 Cell Entry Depends on ACE2 and TMPRSS2 and Is Blocked by a Clinically Proven Protease Inhibitor" (in English). Cell 181 (2): 271–280.e8. doi:10.1016/j.cell.2020.02.052. ISSN 0092-8674. PMC PMC7102627. PMID 32142651. https://www.cell.com/cell/abstract/S0092-8674(20)30229-4.
- Wu, Renyi; Wang, Lujing; Kuo, Hsiao-Chen Dina; Shannar, Ahmad; Peter, Rebecca; Chou, Pochung Jordan; Li, Shanyi; Hudlikar, Rasika et al. (1 June 2020). "An Update on Current Therapeutic Drugs Treating COVID-19" (in en). Current Pharmacology Reports 6 (3): 56–70. doi:10.1007/s40495-020-00216-7. ISSN 2198-641X. PMC PMC7211915. PMID 32395418. https://doi.org/10.1007/s40495-020-00216-7.
- Levi, Marcel; Thachil, Jecko; Iba, Toshiaki; Levy, Jerrold H. (1 June 2020). "Coagulation abnormalities and thrombosis in patients with COVID-19" (in English). The Lancet Haematology 7 (6): e438–e440. doi:10.1016/S2352-3026(20)30145-9. ISSN 2352-3026. PMC PMC7213964. PMID 32407672. https://www.thelancet.com/journals/lanhae/article/PIIS2352-3026(20)30145-9/abstract.
- Nile, S.H.; Nile, A.; Qiu, J. et al. (1 June 2020). "COVID-19: Pathogenesis, cytokine storm and therapeutic potential of interferons" (in en). Cytokine & Growth Factor Reviews 53: 66–70. doi:10.1016/j.cytogfr.2020.05.002. ISSN 1359-6101. PMC PMC7204669. PMID 32418715. https://www.sciencedirect.com/science/article/pii/S1359610120300708.
- Cleary, Simon J.; Pitchford, Simon C.; Amison, Richard T.; Carrington, Robert; Robaina Cabrera, C. Lorena; Magnen, Mélia; Looney, Mark R.; Gray, Elaine et al. (1 November 2020). "Animal models of mechanisms of SARS‐CoV‐2 infection and COVID‐19 pathology" (in en). British Journal of Pharmacology 177 (21): 4851–4865. doi:10.1111/bph.15143. ISSN 0007-1188. PMC PMC7283621. PMID 32462701. https://onlinelibrary.wiley.com/doi/10.1111/bph.15143.
- Lakdawala, Seema S.; Menachery, Vineet D. (29 May 2020). "The search for a COVID-19 animal model" (in en). Science 368 (6494): 942–943. doi:10.1126/science.abc6141. ISSN 0036-8075. https://www.science.org/doi/10.1126/science.abc6141.
- Muñoz-Fontela, César; Dowling, William E.; Funnell, Simon G. P.; Gsell, Pierre-S.; Riveros-Balta, A. Ximena; Albrecht, Randy A.; Andersen, Hanne; Baric, Ralph S. et al. (22 October 2020). "Animal models for COVID-19" (in en). Nature 586 (7830): 509–515. doi:10.1038/s41586-020-2787-6. ISSN 0028-0836. PMC PMC8136862. PMID 32967005. https://www.nature.com/articles/s41586-020-2787-6.
- Rockx, Barry; Kuiken, Thijs; Herfst, Sander; Bestebroer, Theo; Lamers, Mart M.; Oude Munnink, Bas B.; de Meulder, Dennis; van Amerongen, Geert et al. (29 May 2020). "Comparative pathogenesis of COVID-19, MERS, and SARS in a nonhuman primate model" (in en). Science 368 (6494): 1012–1015. doi:10.1126/science.abb7314. ISSN 0036-8075. PMC PMC7164679. PMID 32303590. https://www.science.org/doi/10.1126/science.abb7314.
- Sun, Jing; Zhuang, Zhen; Zheng, Jian; Li, Kun; Wong, Roy Lok-Yin; Liu, Donglan; Huang, Jicheng; He, Jiangping et al. (1 August 2020). "Generation of a Broadly Useful Model for COVID-19 Pathogenesis, Vaccination, and Treatment" (in en). Cell 182 (3): 734–743.e5. doi:10.1016/j.cell.2020.06.010. PMC PMC7284240. PMID 32643603. https://linkinghub.elsevier.com/retrieve/pii/S0092867420307418.
- Rothan, Hussin A.; Byrareddy, Siddappa N. (1 May 2020). "The epidemiology and pathogenesis of coronavirus disease (COVID-19) outbreak" (in en). Journal of Autoimmunity 109: 102433. doi:10.1016/j.jaut.2020.102433. PMC PMC7127067. PMID 32113704. https://linkinghub.elsevier.com/retrieve/pii/S0896841120300469.
- Andrade, E.L.; Bento, A.F.; Cavalli, J.; Oliveira, S.K.; Schwanke, R.C.; Siqueira, J.M.; Freitas, C.S.; Marcon, R. et al. (2016). "Non-clinical studies in the process of new drug development - Part II: Good laboratory practice, metabolism, pharmacokinetics, safety and dose translation to clinical studies". Brazilian Journal of Medical and Biological Research 49 (12): e5646. doi:10.1590/1414-431x20165646. ISSN 1414-431X. PMC PMC5188860. PMID 27982281. http://www.scielo.br/scielo.php?script=sci_arttext&pid=S0100-879X2016001200301&tlng=en.
- "Title 21, Chapter I, Subchapter A, Part 58 Good Laboratory Practice for Nonclinical Laboratory Studies". U.S. Food and Drug Administration. 29 March 2022. https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfcfr/CFRSearch.cfm?fr=58.3. Retrieved 27 May 2022.
- Gibbon, Gerst A. (1 May 1996). "A brief history of LIMS" (in en). Laboratory Automation & Information Management 32 (1): 1–5. doi:10.1016/1381-141X(95)00024-K. https://linkinghub.elsevier.com/retrieve/pii/1381141X9500024K.
- Smith, Peter (2013). Professional website performance: optimizing the front-end and back-end. Indianapolis: Wiley. ISBN 978-1-118-48752-5. OCLC 809561237. https://www.worldcat.org/title/mediawiki/oclc/809561237.
- Lubbers, Peter; Albers, Brian; Salim, Frank (2011) (in en). Pro HTML5 Programming. Berkeley, CA: Apress. doi:10.1007/978-1-4302-3865-2. ISBN 978-1-4302-3864-5. http://link.springer.com/10.1007/978-1-4302-3865-2.
- Sandhu, R.S.; Samarati, P. (1 September 1994). "Access control: principle and practice". IEEE Communications Magazine 32 (9): 40–48. doi:10.1109/35.312842. ISSN 0163-6804. http://ieeexplore.ieee.org/document/312842/.
- Center for Drug Evaluation and Research; Center for Biologics Evaluation and Research (17 October 2019). "M3(R2) Nonclinical Safety Studies for the Conduct of Human Clinical Trials and Marketing Authorization for Pharmaceuticals". U.S. Food and Drug Administration. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/m3r2-nonclinical-safety-studies-conduct-human-clinical-trials-and-marketing-authorization. Retrieved 27 May 2022.
- Bao, Linlin; Deng, Wei; Huang, Baoying; Gao, Hong; Liu, Jiangning; Ren, Lili; Wei, Qiang; Yu, Pin et al. (30 July 2020). "The pathogenicity of SARS-CoV-2 in hACE2 transgenic mice" (in en). Nature 583 (7818): 830–833. doi:10.1038/s41586-020-2312-y. ISSN 0028-0836. http://www.nature.com/articles/s41586-020-2312-y.
- Sun, Shi-Hui; Chen, Qi; Gu, Hong-Jing; Yang, Guan; Wang, Yan-Xiao; Huang, Xing-Yao; Liu, Su-Su; Zhang, Na-Na et al. (1 July 2020). "A Mouse Model of SARS-CoV-2 Infection and Pathogenesis" (in en). Cell Host & Microbe 28 (1): 124–133.e4. doi:10.1016/j.chom.2020.05.020. PMC PMC7250783. PMID 32485164. https://linkinghub.elsevier.com/retrieve/pii/S1931312820303024.
- Huang, Chaolin; Wang, Yeming; Li, Xingwang; Ren, Lili; Zhao, Jianping; Hu, Yi; Zhang, Li; Fan, Guohui et al. (1 February 2020). "Clinical features of patients infected with 2019 novel coronavirus in Wuhan, China" (in en). The Lancet 395 (10223): 497–506. doi:10.1016/S0140-6736(20)30183-5. PMC PMC7159299. PMID 31986264. https://linkinghub.elsevier.com/retrieve/pii/S0140673620301835.
- Sia, Sin Fun; Yan, Li-Meng; Chin, Alex W. H.; Fung, Kevin; Choy, Ka-Tim; Wong, Alvina Y. L.; Kaewpreedee, Prathanporn; Perera, Ranawaka A. P. M. et al. (30 July 2020). "Pathogenesis and transmission of SARS-CoV-2 in golden hamsters" (in en). Nature 583 (7818): 834–838. doi:10.1038/s41586-020-2342-5. ISSN 0028-0836. PMC PMC7394720. PMID 32408338. http://www.nature.com/articles/s41586-020-2342-5.
- Chan, Jasper Fuk-Woo; Zhang, Anna Jinxia; Yuan, Shuofeng; Poon, Vincent Kwok-Man; Chan, Chris Chung-Sing; Lee, Andrew Chak-Yiu; Chan, Wan-Mui; Fan, Zhimeng et al. (3 December 2020). "Simulation of the Clinical and Pathological Manifestations of Coronavirus Disease 2019 (COVID-19) in a Golden Syrian Hamster Model: Implications for Disease Pathogenesis and Transmissibility". Clinical Infectious Diseases: An Official Publication of the Infectious Diseases Society of America 71 (9): 2428–2446. doi:10.1093/cid/ciaa325. ISSN 1537-6591. PMC 7184405. PMID 32215622. https://pubmed.ncbi.nlm.nih.gov/32215622.
- Hoyer, Carolin; Ebert, Anne; Huttner, Hagen B.; Puetz, Volker; Kallmünzer, Bernd; Barlinn, Kristian; Haverkamp, Christian; Harloff, Andreas et al. (1 July 2020). "Acute Stroke in Times of the COVID-19 Pandemic: A Multicenter Study" (in en). Stroke 51 (7): 2224–2227. doi:10.1161/STROKEAHA.120.030395. ISSN 0039-2499. https://www.ahajournals.org/doi/10.1161/STROKEAHA.120.030395.
- Barret, Juan P.; Chong, Si Jack; Depetris, Nadia; Fisher, Mark D.; Luo, Gaoxing; Moiemen, Naiem; Pham, Tam; Qiao, Liang et al. (1 August 2020). "Burn center function during the COVID-19 pandemic: An international multi-center report of strategy and experience" (in en). Burns 46 (5): 1021–1035. doi:10.1016/j.burns.2020.04.003. PMC PMC7151262. PMID 32416984. https://linkinghub.elsevier.com/retrieve/pii/S0305417920302746.
- Huang, Mingxing; Li, Man; Xiao, Fei; Pang, Pengfei; Liang, Jiabi; Tang, Tiantian; Liu, Shaoxuan; Chen, Binghui et al. (12 September 2020). "Preliminary evidence from a multicenter prospective observational study of the safety and efficacy of chloroquine for the treatment of COVID-19" (in en). National Science Review 7 (9): 1428–1436. doi:10.1093/nsr/nwaa113. ISSN 2095-5138. PMC PMC7313782. PMID 34676087. https://academic.oup.com/nsr/article/7/9/1428/5848167.
- D'Ascenzo, Fabrizio; De Filippo, Ovidio; Borin, Andrea; Barbieri, Lucia; Adamo, Marianna; Morici, Nuccia; Truffa Giachet, Alessandra; Iannaccone, Mario et al. (1 June 2021). "Impact of COVID-19 pandemic and infection on in hospital survival for patients presenting with acute coronary syndromes: A multicenter registry" (in en). International Journal of Cardiology 332: 227–234. doi:10.1016/j.ijcard.2021.03.063. PMC PMC8006512. PMID 33794235. https://linkinghub.elsevier.com/retrieve/pii/S0167527321005672.
- Yu, Chao; Zhou, Miao; Liu, Yang; Guo, Tinglin; Ou, Chongyang; Yang, Liye; Li, Yan; Li, Dongliang et al. (31 December 2020). "Characteristics of asymptomatic COVID-19 infection and progression: A multicenter, retrospective study" (in en). Virulence 11 (1): 1006–1014. doi:10.1080/21505594.2020.1802194. ISSN 2150-5594. PMC PMC7550018. PMID 32722990. https://www.tandfonline.com/doi/full/10.1080/21505594.2020.1802194.
- Simpson, Shmona; Kaufmann, Michael C; Glozman, Vitaly; Chakrabarti, Ajoy (1 May 2020). "Disease X: accelerating the development of medical countermeasures for the next pandemic" (in en). The Lancet Infectious Diseases 20 (5): e108–e115. doi:10.1016/S1473-3099(20)30123-7. PMC PMC7158580. PMID 32197097. https://linkinghub.elsevier.com/retrieve/pii/S1473309920301237.
This presentation is faithful to the original, with only a few minor changes to presentation, grammar, and spelling. In some cases important information was missing from the references, and that information was added. In this version, a little bit of additional context and citation is given in the Background concerning non-clinical animal studies vs. clinical animal studies. The Methods section appears at the end of the original article; it has been moved up above Results for this version.