Journal:Mini-review of laboratory operations in biobanking: Building biobanking resources for translational research

From LIMSWiki
Jump to navigationJump to search
Full article title Mini-review of laboratory operations in biobanking: Building biobanking resources for translational research
Journal Frontiers in Public Health
Author(s) Cicek, Mine S.; Olson, Janet E.
Author affiliation(s) Mayo Clinic
Primary contact cicek dot mine at mayo dot edu
Editors Kilbourne, Amy
Year published 2020
Volume and issue 8
Article # 362
DOI 10.3389/fpubh.2020.00362
ISSN 2296-2565
Distribution license Creative Commons Attribution 4.0 International
Download (PDF)


Biobanks have become integral to improving population health. We are in a new era in medicine as patients, health professionals, and researchers increasingly collaborate to gain new knowledge and explore new paradigms for diagnosing and treating disease. Many large-scale biobanking efforts are underway worldwide at the institutional, national, and even international level. When linked with subject data from questionnaires and medical records, biobanks serve as valuable resources in translational research. A biobank must have high-quality biospecimens that meet researcher's needs. Biobank laboratory operations require an enormous amount of support, from lab and storage space, information technology expertise, and a laboratory information management system to logistics for biospecimen tracking, quality management systems, and appropriate facilities. A paramount metric of success for a biobank is the concept of every biospecimen coming to the repository belonging to a participant who has something to contribute to research for a healthier future. This article will discuss the importance of biorepository operations, specific to the collection and storage of participants' materials. Specific focus will be given to maintaining the quality of biospecimens, along with the various levels of support biorepositories need to fulfill their purpose and ensure the integrity of each biospecimen is maintained.

Keywords: disaster and risk management, biobanking and biorepositories, laboratory information management system (LIMS), biorepository operations, biospecimen research


In the past few decades, as technology and informatics has expanded, scientific research has moved from laboratory-based discovery to translational research, seeking to identify underlying biologic causes for disease and offer personalized treatment for patients. Large numbers of biospecimens, procured from study populations that can be followed across time, have become the critical component for translational research, and the twenty-first century has shown the emergence of biobanks around the world. There are now both large population-based biobanks, and patient-based biobanks, including the Kadoorie Biobank in China[1], LifeGene in Sweden[2], CONOR and MoBa in Norway[3][4], Auria in Finland[5], UK Biobank[6], Estonian Biobank[7], BioBank Japan[8], Korean Biobank[9][10], and the Taiwan Biobank[11], as well as the Million Veteran Program[12] and All of Us Biobank[13] in the United States. Biobanks must collect, process, store, and disseminate biologic material (biospecimens) with appropriate and complete annotation in order to meet the needs of future translational researchers. A successful translational research program depends on the existence of biobanks with solid infrastructure that can provide the needed consistency in specimen collection and tracking, quality control (QC) management, and biospecimen storage, as well as disaster recovery plans and long-term financial stability. These key elements are building blocks for any biobank to be successful and for translational research to thrive. Guidelines have been established for best practices in biobanking and for biospecimen resource creation by both the National Cancer Institute (NCI)[14] and the International Society for Biological and Environmental Repositories (ISBER).[15] Here we provide an overview of important considerations for biorepository operations (Figure 1), defined herein as operational groups that support biospecimen collection and storage across multiple biobanks.

Fig1 Cicek FrontPubHealth2020 8.jpg

Figure 1: An overview of important considerations for biorepository operations

Biobank planning

Prior to initiation of a new biobank collection, the biorepository operational team must obtain an understanding of the new biobank's goals, subject population(s), collection and processing protocols, storage requirements, pre-analytical data collection need and long term monitoring requirements. The biobank should have a clear plan to ensure that it can successfully meet its own collection goals as well as anticipated needs of both current and future translational researchers. The biorepository operation team's role begins early on at the protocol design stage, before the first participant volunteers to take part in the biobank. Operational procedures must be established and tested to ensure each biospecimen will be collected and processed properly and in a timely manner to guarantee its integrity. The integrity of a biospecimen refers to its consistent processing to ensure all collected material remains intact and usable for the purpose for which it was collected. This integrity is tracked by data that is captured at each step of the collection and storage process. It allows for a history of the biospecimen to be recorded to ensure data that will eventually be collected by researchers results in both meaningful and consistent results, and to ensure that nothing happened during its processing or storage that might alter these results. The burden of responsibility resides equally on the staff of the biorepository and biobank (study) to conduct the collection site feasibility assessment, establish standard operating procedures (SOPs), and train personnel involved in biospecimen procurement.

SOP creation

SOPs should be developed to ensure a high-quality biospecimen is collected and consistent handling methods are used to reduce pre-analytical variables, including pre-processing time, shipping temperature, centrifugation speed and temperature, processing time, and storage temperature. Some SOPs fall to the biobank collection team, but many fall to the biorepository staff to implement, including biospecimen handling, laboratory processing, shipping and receiving methodology, documentation within a record management system, maintenance support of equipment, and facility security.[16] Moore et al. reported in 2011 that many factors in biospecimen handling can affect downstream applications, and as such, all published results from experiments should also include a report on how biospecimens were handled during collection, otherwise known as BRISQ (Biospecimen Reporting for Improved Study Quality).[17]

The biorepository operations team should assist the biobank team to evaluate collection tube options—and test if necessary—to determine the appropriate collection protocol which will allow the biospecimens collected to give the optimal desired final product for the goal of the biobank. Processing instructions and allowable time post-collection should be defined clearly in SOPs based on literature and manufacturer's recommendations. In order to maintain molecular stability, biospecimens should be stabilized or processed as soon as possible after collection.[18] However, different collected biospecimens will inherently have different processing requirements. Elliott et al., for example, describe testing done prior to the inception of the collection of UK Biobank biospecimens.[6] One test examined the length of time biospecimens could be stored at 4°C before detectable differences in important analytes arose. Results demonstrated that a time window of 24 hours showed no differences, but at 36 hours, differences were present. Thus, the final SOP was written to reflect those findings, stating that the desired time to cryopreservation of aliquots should be no more than 24 hours after collection.[6] Once established, all SOPs should be reviewed annually and updated as needed.

Biobank staff training

Biospecimen integrity can be maximized by a biobank's adherence to established SOPs as described above, along with properly training its staff. Staff training and periodic review of the overall process is especially important for biobanks that rely on multiple accrual sites for initial patient contact, specimen procurement, and pre-shipment processing, where variations in procedures could happen based on site infrastructure. Training of biobank accrual site staff should include a super-trainer who would serve to train new staff, in anticipation of staff turnover throughout the project. Web-based training modules can also be helpful to biobank accrual sites, to provide continuing support and information regarding SOP updates. Remote ongoing support can be established via call center, where questions are triaged to the appropriate personnel to be answered in a timely manner. Finally, dry runs can be executed as part of onboarding new collection sites, to ensure the process flowed smoothly through shipment prior to collection. This model has resulted in highly successful accrual of quality biospecimens in other biobanks.

Biospecimen workflow and data management

Biorepository operations' daily activity depends on proper bioscpecimen collection and tracking, today commonly facilitated by a laboratory information management system (LIMS). Having a well-established LIMS and other other information technology (IT) solutions in place is critical to ensure a collected biospecimen's data integrity, as well as its overall quality. The workflow of a biobank can be managed by a LIMS, where each collected biospecimen is linked to the correct patient record, from the time of bioscpecime procurement to its shipment, processing, and final long-term storage.[18]

Indeed, for large biobanks, there is an absolute requirement for a LIMS for biospecimen tracking. Methodology and use of an integrated LIMS, along with mode of information entry, can be accomplished in a variety of ways. For example, the Estonian Biobank reports having a recruiter fill out the questionnaire with the participant at time of donation, creating a data record.[7] When required fields were not filled in properly, the recruiter was alerted in real-time, allowing immediate correction. This is one way to ensure all information is both complete and accurate. Using a different method, the UK Biobank assigns a unique barcode to each biospecimen and scans it into their information management system at the assessment center, linking it with the participant's unique ID, which was assigned during enrollment. In this approach the barcoded tubes are not preassigned to a subject to avoid participant identification errors and prevent empty blood collection tubes being logged in. Scanning in barcodes at the time of collection also adds a date- and timestamped record into the UK Biobank's system, so reception time can be recorded automatically rather than manually.[6] Biobank Japan has independent medical coordinators who collect the information and then anonymize the specimens at the collection site.[8] Thus, information management and other IT systems and how they are used can differ across biobanks, but those systems are always crucial to ensure accurate record keeping during biospecimen procurement.

Post-collection processing

After biospecimens are collected, protocols may require some biospecimens undergo minimal processing to ensure their stability; other biospecimens may be stored at a specified temperature before being shipped to a processing center or to the main biobank site. Any processing at collection sites should be documented in the information management system, including time of processing and any problems or variations from the published SOP. Methods and timing for shipment from accrual sites to the biobanks also vary based on the stability of the biospecimen type. Biospecimen tracking through the system remains critical, including tracking its arrival time and temperature upon delivery. For example, biospecimens in the Kadoorie Biobank are placed at 4°C for a few hours until processing, then they are stored at −40°C for three to four months, before being couriered on dry ice to a central blood repository for long-term storage at −80°C.[1] The Million Veteran Program ships whole blood to a processing center for same-day DNA extraction.[12] Biospecimens collected for the UK Biobank have minimal processing done at collection sites and are stabilized at the appropriate temperatures before being sent day-of-collection to high-throughput sites for completion of processing.[6]

Biobanks often use a commercial courier for overnight specimen shipments. Commercial couriers are the preferred method of transportation for biospecimens as they are trained with their own SOPs in temperature control and expedited specimen delivery. Upon arrival at the biobank, each package containing biospecimens must go through a quality check for temperature integrity and package damage. The biospecimens actually received must be reconciled with the attached requisition form. All deviations observed should be recorded in the LIMS.

The quality of a biobank's operations should also be measured by laboratory infrastructure. Large biobanks will have to invest on instrumentation to minimize human errors and automate biospecimen processing and storage.[12][19] Robotics, if appropriate, can provide varying advantages such as reducing cost, improving reproducibility, and providing a better overall quality assurance. Robotics also can be helpful in biospecimen annotation and tracking in the LIMS. Robotic retrieval of biospecimens can also ensure temperature stability.

The goal of biorepository operations management is to maintain the highest quality specimen available for translational research. Protocol deviations result in preanalytical variability that may impact downstream use of biospecimens.[20] In 2000, Narayanan published a detailed review of three areas of pre-analytic variability: physiologic, specimen collection, and influence of interference factors that need to be considered during biospecimen collection.[21] Clearly some of the burden of minimizing pre-analytic variables will fall upon the biorepository. Numerous articles and reviews have been published reviewing and addressing best practices for biobanks—specifically in the areas of biospecimen collection and processing, tracking, and storage[18][21][22][23]—and are available as resources. Quality management can also be maintained through the use of routine testing and QC metrics of the processed specimens, as well as the through the use of laboratory robotics. Any variance from SOPs as well as regular QC and maintenance records should be documented. Finally, the biorepository staff should conduct periodic review of the biobanking literature to ensure that SOPs are updated with the latest advancements found in newly published reports and technology demos.

Even with all of this standardization, each biorepository's operational teams will vary somewhat in their methods. However, biospecimens from different biobanks must have common requirements and metrics to ensure the quality of all biospecimens at all biobanks, especially when procedures may vary. The Biorepository Accreditation Program (BAP), created by the College of American Pathologists (CAP), serves to provide requirements for standardization of biospecimen processes that will result in high-quality biospecimens.[24] Accreditation is a two-year cyclical process, involving one year of on-site inspections by certified peers who review documents, observe practices, and ask questions of staff, and then one year of self-reported inspections. Checklists containing standards are evaluated, and, though exact protocols may vary from lab to lab, all must be completed. Specific personnel qualifications, training, and competency assessments also must be met.[24] Checklists and findings are reviewed and recommendations are given to fix deficiencies found. As of 2018, 53 biorepositories are fully CAP BAP accredited, with many more in the review phase.

Disaster preparedness

Biobanks represent an irreplaceable asset, and consideration should be given toward preparation for possible urgent circumstances such as fires, floods, tornados, or hurricanes.[25][26][27][28] The COVID-19 pandemic of 2019–2020 represents another urgent situation, providing a unique challenge of a different sort. Instead of reacting to a natural disaster that may involve structural damages, or power outages, this pandemic affects how biospecimens are collected and processed. Decisions have needed to be made to assess if biospecime collection could even still safely continue. Would at-home collections and shipments still be possible, particularly in regards to ensuring biospecimen integrity? Planning has been essential towards protecting the time and value of yet-to-be-collected biospecimens and the people donating them as well.

ISBER recommends that biobanks have an emergency management plan and a disaster contingency plan.[29] The establishment of a crisis management team is needed to identify and assess risks specific to each biobank and to create a risk response as part of an established contingency plan, should an identified risk occur.[29] For example, could virtual data collection be done during a pandemic? How could biospecimen integrity be ensured if at-home collection was performed? How can shipments protocols be ensured? For a more in-depth review, see Parry-Jones et al.[29], who describe how a crisis management plan can be developed. Along with this plan, storing biospecimens along with electronic records at two geographically separated locations is another way to secure biospecimens and ensure a biobank can withstand a disaster without losing the complete collection.


Finally, long term biobank sustainability must be considered. The NCI Biorepositories and Biospecimen Research Branch developed a web-based application (the Biobank Economic Modeling Tool) to enable determination of the accurate cost recovery fee to help with long-range planning.[30] To keep the biobank running long-term, it is essential that biobanks accurately assess financial costs for distribution of biospecimens. Also, planning to aid in the operational efficiency of a biobank is critical. According to Watson et al.[31], this includes input efficiency (patient enrollment and biospecimen accrual), internal efficiency (optimizing processing, balancing resources to support retrospective questions while not storing beyond what is anticipated to be useful), and offering more data elements (like records of pre-analytic variables). The more biospecimens that are requested from a biobank, the more financial compensation it can receive. Securing the social adaptability of the biobank will allow it to become a resource for all to use. Having an ethics review board that examines biobank and research projects, commits to good practices, and remains open with the public about usage and returned results (if appropriate) is critical.


Biobanks have become integral to improving population health. We are in a new era in medicine as patients, health professionals, and researchers increasingly collaborate to gain new knowledge and explore new paradigms for diagnosing and treating disease. Many large-scale biobanking efforts are underway worldwide at the institutional, national, and even international level. When linked with subject data from questionnaires and medical records, biobanks serve as valuable resources in translational research. A biobank must have high-quality biospecimens that meet researcher's needs. Biorepository operations require an enormous amount of support, from lab and storage space, information technology expertise, and a LIMS to logistics for biospecimen tracking, quality management systems, and appropriate facilities (Table 1). A paramount metric of success for a biobank is the concept that every biospecimen coming to the repository belongs to a participant who has something to contribute to research for a healthier future.

Table 1. Running a biorepository is a complex operation and requires multiple components to work in harmony to be successful.
Components of a biobank Purpose
Pre-planning porcess To understand the goals of the biobank
Standard operating procedures (SOPs) To create protocol documents to ensure laboratory procedures are trained on and followed by all staff and to minimize pre-analytic viability in order to have quality specimens that meet the goals of the biobank
Biobank staff training To ensure all staff involved in specimen collection, processing, and storage follow SOPs consistently
Laboratory information management system (LIMS) To track a specimen from collection through processing and storage, and to link patients to biospecimen records throughout the lifecycle of a biospecimen
Disaster preparedeness plan To create a plan to ensure biobank operations are secure and keep biospecimens safe from unexpected disasters
Sustainability To create a plan to ensure that biobank operations are financially stable and remain a long-term resource


Author contributions

MC: corresponding author, responsible for conception and design of the manuscript, drafted and edited, and responsible for answering questions and revisions. JO: equally accountable for editing and responsible for answering questions. All authors contributed to the article and approved the submitted version.


his manuscript was supported by the Mayo Clinic Center for Individualized Medicine.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.


  1. 1.0 1.1 Chen, Z.; Chen, J.; Collins, R. et al. (2011). "China Kadoorie Biobank of 0.5 million people: Survey methods, baseline characteristics and long-term follow-up". International Journal of Epidemiology 40 (6): 1652-66. doi:10.1093/ije/dyr120. PMC PMC3235021. PMID 22158673. 
  2. Almqvist, C.; Adami, H.-O.; Franks, P.W. et al. (2011). "LifeGene--a large prospective population-based study of global relevance". International Journal of Epidemiology 26 (1): 67–77. doi:10.1007/s10654-010-9521-x. PMC PMC7087900. PMID 21104112. 
  3. Naess, O.; Søgaard, A.J.; Arnesen, E. et al. (2008). "Cohort profile: Cohort of Norway (CONOR)". International Journal of Epidemiology 37 (3): 481–5. doi:10.1093/ije/dym217. PMC PMC2409050. PMID 17984119. 
  4. Rønningen, K.S.; Paltiel, L.; Meltzer H.M. et al. (2006). "The biobank of the Norwegian Mother and Child Cohort Study: A resource for the next 100 years". European Journal of Epidemiology 21 (8): 619–24. doi:10.1007/s10654-006-9041-x. PMC PMC1820840. PMID 17031521. 
  5. "Auria Biobank". Retrieved 20 Novemember 2019. 
  6. 6.0 6.1 6.2 6.3 6.4 Elliott, P.; Peakman, T.C.; UK Biobank (2008). "The UK Biobank sample handling and storage protocol for the collection, processing and archiving of human blood and urine". International Journal of Epidemiology 37 (2): 234–44. doi:10.1093/ije/dym276. PMID 18381398. 
  7. 7.0 7.1 Leitsalu, L.; Haller, T.; Esko, T. et al. (2015). "Cohort Profile: Estonian Biobank of the Estonian Genome Center, University of Tartu". International Journal of Epidemiology 44 (4): 1137-47. doi:10.1093/ije/dyt268. PMID 24518929. 
  8. 8.0 8.1 Nagai, A.; Hirata, M.; Kamatani, Y. et al. (2017). "Overview of the BioBank Japan Project: Study design and profile". Journal of Epidemiology 27 (3S): S2–S8. doi:10.1016/ PMC PMC5350590. PMID 28189464. 
  9. Lee, J.-E.; Kim, J.-H.; Hong, E.-J. et al. (2012). "National Biobank of Korea: Quality control Programs of Collected-human Biospecimens". Osong Public Health and Research Perspectives 3 (3): 185–9. doi:10.1016/j.phrp.2012.07.007. PMC PMC3738702. PMID 24159512. 
  10. Lee, S.; Jung, P.E.; Lee, Y. (2016). "Publicly-funded biobanks and networks in East Asia". SpringerPlus 5 (1): 1080. doi:10.1186/s40064-016-2723-2. PMC PMC4945521. PMID 27462528. 
  11. Lin, J.-C.; Chen, L.-K.; Hsiao, W.W.-W. et al. (2019). "Next Chapter of the Taiwan Biobank: Sustainability and Perspectives". Biopreservation and Biobanking 17 (2): 189–97. doi:10.1089/bio.2018.0119. PMID 30715914. 
  12. 12.0 12.1 12.2 Gaziano, J.M.; Concato, J.; Brophy, M. et al. (2016). "Million Veteran Program: A mega-biobank to study genetic influences on health and disease". Journal of Clinical Epidemiology 70: 214-23. doi:10.1016/j.jclinepi.2015.09.016. PMID 26441289. 
  13. All of Us Research program Investigators; Denny, J.C.; Rutter, J.L. et al. (2019). "The "All of Us" Research Program". New England Journal of Medicine 381 (7): 668–76. doi:10.1056/NEJMsr1809937. PMID 31412182. 
  14. Biorepositories & Biospecimen Research Branch (BBRB) (17 December 2018). "Best Practices". National Cancer Institute. Retrieved 20 November 2019. 
  15. Campbell, L.D.; Astrin, J.J.; DeSouza, Y. et al. (2018). "The 2018 Revision of the ISBER Best Practices: Summary of Changes and the Editorial Team's Development Process". Biopreservation and Biobanking 16 (1): 3–6. doi:10.1089/bio.2018.0001. PMC PMC5846567. PMID 29393664. 
  16. Vaught, J.; Lockhart, N.C. (2012). "The evolution of biobanking best practices". Clinica Chimica Acta 413 (19–20): 1569–75. doi:10.1016/j.cca.2012.04.030. PMC PMC3409343. PMID 22579478. 
  17. Moore, H.M.; Kelly, A.; Jewell, S.D. et al. (2011). "Biospecimen Reporting for Improved Study Quality". Biopreservation and Biobanking 9 (1): 57–70. doi:10.1089/bio.2010.0036. PMC PMC3142856. PMID 21826252. 
  18. 18.0 18.1 18.2 Malm, J.; Fehniger, T.E.; Danmyr, P. et al. (2013). "Developments in biobanking workflow standardization providing sample integrity and stability". Journal of Proteomics 95: 38–45. doi:10.1016/j.jprot.2013.06.035. PMID 23856607. 
  19. Peakman, T.C.; Elliott, P. (2008). "The UK Biobank sample handling and storage validation studies". International Journal of Epidemiology 37 Suppl 1: i2–i6. doi:10.1093/ije/dyn019. PMID 18381389. 
  20. Betsou, F.; Barnes, R.; Burke, T. et al. (2009). "Human biospecimen research: Experimental protocol and quality control tools". Cancer Epidemiology, Biomarkers and Prevention 18 (4): 1017–25. doi:10.1158/1055-9965.EPI-08-1231. PMID 19336543. 
  21. 21.0 21.1 Narayanan, S. (2000). "The preanalytic phase. An important component of laboratory medicine". American Journal of Clinical Pathology 113 (3): 429–52. doi:10.1309/C0NM-Q7R0-LL2E-B3UY. PMID 10705825. 
  22. Shabihkhani, M.; Lucey, G.M.; Wei, B. et al. (2014). "The procurement, storage, and quality assurance of frozen blood and tissue biospecimens in pathology, biorepository, and biobank settings". Clinical Biochemistry 47 (4–5): 258–66. doi:10.1016/j.clinbiochem.2014.01.002. PMC PMC3982909. PMID 24424103. 
  23. Holland, N.T.; Smith, M.T.; Eskenazi, B. et al. (2003). "Biological sample collection and processing for molecular epidemiological studies". Mutation Research 543 (3): 217–34. doi:10.1016/s1383-5742(02)00090-x. PMID 12787814. 
  24. 24.0 24.1 McCall, S.J.; Branton, P.A.; Blanc, V.M. et al. (2018). "The College of American Pathologists Biorepository Accreditation Program: Results from the First 5 Years". Biopreservation and Biobanking 16 (1): 16–22. doi:10.1089/bio.2017.0108. PMC PMC5824654. PMID 29394087. 
  25. Bjugen, R.; Hansen, J. (2013). "Learning by Erring: Fire!". Biopreservation and Biobanking 11 (4): 202–5. doi:10.1089/bio.2013.0020. PMID 24845585. 
  26. Roswall, N.; Jalkjær, J.; Overvad, K. et al. (2013). "Measures taken to restore the Danish Diet, Cancer and Health Biobank after flooding: A framework for future biobank restorations". Biopreservation and Biobanking 11 (4): 206–10. doi:10.1089/bio.2013.0029. PMID 24845586. 
  27. Morrin, H.R.; Robinson, B.A. (2013). "Sustaining a biobank through a series of earthquake swarms: Lessons learned from our New Zealand experience". Biopreservation and Biobanking 11 (4): 211–5. doi:10.1089/bio.2013.0033. PMID 24845587. 
  28. Mintzer, J.L.; Kronenthal, C.J.; Kelly, V. et al. (2013). "Preparedness for a natural disaster: How Coriell planned for hurricane Sandy". Biopreservation and Biobanking 11 (4): 216–20. doi:10.1089/bio.2013.0035. PMID 24845588. 
  29. 29.0 29.1 29.2 Parry-Jones, A.; Hansen, J.; Simeon-Dubach, D. et al. (2017). "Crisis Management for Biobanks". Biopreservation and Biobanking 15 (3): 253-263. doi:10.1089/bio.2016.0048. PMID 27977307. 
  30. Odeh, H.; Miranda, L.; Rao, A. et al. (2015). The Biobank Economic Modeling Tool (BEMT): Online Financial Planning to Facilitate Biobank Sustainability. 13. pp. 421–9. doi:10.1089/bio.2015.0089. PMC PMC4696440. PMID 26697911. 
  31. Watson, P.H.; Nussbeck, S.Y.; Carter, C. et al. (2014). A framework for biobank sustainability. 12. pp. 60–8. doi:10.1089/bio.2013.0064. PMC PMC4150367. PMID 24620771. 


This presentation is faithful to the original, with only a few minor changes to presentation. A few grammar and spelling errors were also corrected. In some cases important information was missing from the references, and that information was added.