Template:COVID-19 Testing, Reporting, and Information Management in the Laboratory/Workflow and information management for COVID-19 (and other respiratory diseases)/Additional benefits and challenges of informatics in disease testing and public health

From LIMSWiki
Jump to navigationJump to search

4.3 Additional benefits and challenges of laboratory informatics in disease testing and public health

COVID-19 is at the forefront of the consciousness of humanity, by and large, and the informatics tools we implement for managing, treating, and surveilling the disease are of great import. From disease databases to electronic health records, from bioinformatics tools for peptide and protein modeling to laboratory tools such as LIMS and LIS, we continue to fight back against the threat of the SARS-CoV-2 virus. Yet despite the gravity of the pandemic, this is neither the first nor the last time laboratory and scientific informatics will play a positive role in testing for disease and improving public health outcomes.

Health informatics technology, when used responsibly, has already proven to be useful in studying and treating contagious diseases. In a 2013 research paper published in the journal BMJ Quality & Safety, El-Kareh et al. analyzed and described the state of diagnostic health information technology (HIT). They noted that without the aid of HIT, clinicians are more error-prone, leaving them "vulnerable to fallible human memory, variable disease presentation, clinical processes plagued by communication lapses, and a series of well-documented ‘heuristics,’ biases, and disease-specific pitfalls."[1] Appropriate, well-designed HIT systems are capable of helping clinicians and laboratorians by providing more timely access to information, improved communication, better clinical reasoning and decision making, and improved workflows, as well as a reduction in diagnostic errors, and, as a result, improved patient safety and health outcomes.[2]

From a public health perspective, the application of informatics to disease surveillance, reporting, and health habit promotion is also vital. Winters-Miner et al. note in particular the value of using informatics tools and methods to implement predictive analytics and data mining into public health. They use disease prevention and biosurveillance as major examples. We could, for example "analyze large populations of people to quantify risks related to public health, and help physicians to develop intervention programs for those patients at highest risk of some ailment or medical condition."[3] Additionally, through the use of syndromic surveillance systems (tools aiding in the detection of indicators leading up to disease diagnosis for individuals and populations[4]), they suggest that outbreaks can be better detected at local and national levels, and public health measures can be better implemented, increasing public awareness and hindering the spread of disease.[3]

In the clinical laboratory, informatics systems have been influencing workflow improvements and improved service delivery for more than five decades.[5] And while improvements have been seen in the laboratory from not only the introduction of computerized systems[1][2][6] but also the realization of quality control[7] and point-of-care testing[8], more challenges remain. For example, quality management in the laboratory is still often a manual, time-consuming activity. While the LIMS and LIS have some tools to assist with this task, the inclusion of laboratory analytics and business intelligence tools into those systems may lead to even further improvements in quality and efficiency in the lab.[9] And in the realm of point-of-care testing, oversight and control of instruments can be lost when connectivity and training is lacking. Proper interfacing of these lab instruments could lead to improvements in those areas, says Siemens Healthineers' Daniel Gundler. "Maintaining POC instruments and overseeing the operators performing POC tests would be much easier if all the information and data from each instrument were accessible through one user interface in which coordinators could manage both the instruments and operators."[10]

4.3.1 System interoperability

Electronic medical record.jpg

System interoperability also poses benefits and challenges to clinical disease testing and prevention. Interoperability is defined as “the ability of different information systems, devices and applications (‘systems’) to access, exchange, integrate and cooperatively use data in a coordinated manner” to ensure timely, portable information and improved health outcomes.[11] Improving interoperability among clinical informatics systems is recognized as an important step towards improving health outcomes.[12][13] The National Academies of Sciences, Engineering, and Medicine had much to say on this topic in their 2015 publication Improving Diagnosis in Health Care[2]:

Improved interoperability across different health care organizations—as well as across laboratory and radiology information systems—is critical to improving the diagnostic process. Challenges to interoperability include the inconsistent and slow adoption of standards, particularly among organizations that are not subject to EHR certification programs, as well as a lack of incentives, including a business model that generates revenue for health IT vendors via fees associated with transmitting and receiving data.

In particular, they discuss an additional concern, one that still causes issues today: interfaces between electronic health records (EHR) and the laboratory and other clinical information systems that feed medical diagnostic information into the EHRs[2]:

Additionally, the interface between EHRs and laboratory and radiology information systems typically has limited clinical information, and the lack of sufficiently detailed information makes it difficult for a pathologist or radiologist to determine the proper context for interpreting findings or to decide whether diagnostic testing is appropriate. For example, one study found that important non-oncological conditions (such as Crohn’s disease, human immunodeficiency virus, and diabetes) were not mentioned in 59 percent of radiology orders and the presence of cancer was not mentioned in 8 percent of orders, demonstrating that the complete patient context is not getting received. Insufficient clinical information can be problematic as radiologists and pathologists often use this information to inform their interpretations of diagnostic testing results and suggestions for next steps. In addition, the Centers for Disease Control and Prevention’s Clinical Laboratory Improvement Advisory Committee (CLIAC) expressed concern over the patient safety risks regarding the interoperability of laboratory data and display discrepancies in EHRs. They recommended that laboratory health care professionals collaborate with other stakeholders to “develop effective solutions to reduce identified patient safety risks in and improve the safety of EHR systems” regarding laboratory data.

In fact, interoperability issues have come up during the global laboratory response to the COVID-19 pandemic. In early April 2020, a report from Nature revealed that academic research laboratories wanting to assist with COVID-19 testing efforts have at times been stymied by the incompatibility between academic informatics systems and hospital EHRs. Not only do hospitals use EHRs of differing types, but many of those EHRs were not designed to talk to other EHRs, let alone to academic and research laboratories' informatics systems. Combine this with strict account procedures and the costs of developing interfaces on-the-fly, more than a few medical systems have turned away the offer of help from academic and research labs.[14] As it turns out, HL7- and other standard-based interfaces have long been expensive for many vendors to implement[15], the cost justified typically when high volumes of samples are involved. Additionally, in more normal, non-pandemic circumstances, the requirement to interface with EHRs and hospital information systems (HIS) is almost exclusively found in the LIS and LIMS used in patient settings, i.e., in the hospitals, medical offices, and laboratories catering to diagnosing disease in patients. Academic labs have not been equipped at any level (software, hardware, or personnel) to do high volume clinical testing, nor have they had reason to ensure their informatics systems can interface with clinical systems.

Interoperability benefits and challenges show up elsewhere too. Take for example the value of phenotypes, a representation of the genetic analysis of the collective observable traits of an organism, traits caused by the interaction of its genome with the environment. The value of patient phenotyping data is increasingly useful in the fight against known and novel viruses, as well as a broad variety of non-viral diseases. As Ausiello and Shaw note, in order for medicine to advance and produce improved patient outcomes, "traditional clinical information must be combined with genetic data and non-traditional phenotypes and analyzed in a manner that yields actionable insights into disease diagnosis, prevention, or treatment."[16] Whether it's identifying "the measurable phenotypic characteristics of patients that are most predictive of individual variation" in treatment outcomes for chronic pain[17] or COVID-19[18][19], phenotypes have utility in the clinical sector.

Here again interoperability between EHRs and laboratory informatics systems comes into play. In a 2019 paper published by Zhang et al. in nph Digital Medicine, the topic of extracting patient phenotypes from laboratory test results fed into EHRs is addressed.[20] The authors state that one of the more difficult aspects of their research is that while "[l]aboratory tests have broad applicability for translational research ... EHR-based research using laboratory data have been challenging because of their diversity and the lack of standardization of reporting laboratory test results." They add[20]:

Despite the great potential of EHR data, patient phenotyping from EHRs is still challenging because the phenotype information is distributed in many EHR locations (laboratories, notes, problem lists, imaging data, etc.) and since EHRs have vastly different structures across sites. This lack of integration represents a substantial barrier to widespread use of EHR data in translational research.

The answer to the clinical and laboratory interoperability question is unclear. A 2019 article in the American Association for Clinical Chemistry's CLN Stat addressed remaining roadblocks, including lack of standards development, data quality issues, clinical data matching, lack of incentivizing health IT optimization, text-based reporting formats, differences in terminology, and HL7 messaging issues. They add that proposals from the Office of the National Coordinator for Health Information Technology (ONC) and the Centers for Medicare and Medicaid Services include possible fixes such as standardized application programming interfaces (API). They also note that middleware may pick up the slack in connecting more laboratory devices, rather than depending on the LIS to handle all the interfacing.[21] On a more positive note, the Office of the National Coordinator for Health Information Technology (ONC) updated its Interoperability Standards Advisory (ISA) Vocabulary/Code Set/Terminology page in November to better "highlight critical public health interoperability needs on COVID-19 in an easily accessible way." Replying to the ONC, HIMSS's Senior Director Jeff Coughlin goes on to add[22]:

There is a growing need to consider data exchange for home settings and considerations around device interoperability. There are a number of applications in use and this setting requires work across a number of systems (emergency medical services, hospital electronic health records, telemedicine system [synchronous and asynchronous] and, remote patient monitoring and device management). ISA should provide guidance on specific standards to assist in exchange with this setting."

Even so, it remains obvious that more work needs to be done in the development and standard use of clinical and laboratory informatics applications if the promise of personalized medicine and the need for improved disease testing and response are to be fulfilled. In particular, how we responsibly protect personal health information while putting its anonymized variants to beneficial use for disease testing and prevention remains a critical question that must be solved in order to better prepare for the next COVID-19.

4.3.2 Contact tracing

What is contact tracing COVID-19.png

Contact tracing is a public health process that involves the determination of who a patient had specific contact with while infectious, then informing, supporting, and maintaining contact with those effected individuals in order to reduce the spread of an infection.[23] The seeds of contact tracing may go as far back as late fifteenth and early sixteenth century efforts to control infection rates of syphilis among prostitutes in part of Europe.[24] However, the practice was adopted more vigorously in the late nineteenth and early twentieth century in Great Britain for tracking cases of communicable diseases such as measles[25] and sexually transmitted diseases like syphilis and gonorrhea.[26] Further inroads were made in the United States in the early 1930s with its efforts to reduce sexually transmitted diseases among U.S. troops.[27]

Throughout it all, contract tracing has been predominately a manual effort. However, the technological age has brought with it both social promise and privacy questions in regards to the application of informatics to contact tracing. In the mid-2000s, the use of Wi-Fi and RFID technology in Singapore hospital settings "to provide an alternative to tedious and error-prone manual contact tracing" was being reported in academic literature.[28] Since then, the use of informatics tools in contact tracing has increased, seeing practical application in combating severe acute respiratory syndrome (SARS)[29], Ebola[30], and tuberculosis.[31] As such, it should not be surprising that early in the COVID-19 pandemic, contract tracing was also being discussed within the context of both manual and digital tracing efforts.[23][32][33][34]

Contact tracing using digital tools, however, comes with both benefits and challenges.[35][36] Digital contact tracing tools have the potential to make contract tracing quicker and more efficient than manual processes. People can forget who they may have been in close contact with, for example, but with digital tracing, past location data can be mined to provide a clearer picture. As a result, with more accurate data, local and state policy-making efforts can be more nimbly tailored to current needs, and the effectiveness of those efforts can be better monitored.[31][28][35][37] However, several challenges also exist, primarily with social and cultural perceptions of the technology's acceptability and the privacy concerns surrounding it.

Social acceptability and trust around the world

The social acceptability and trust put into contact tracing applications and the governments that use them is of significant concern. An April 2020 "blitz" survey of the Flemish Region of Belgium showed that only 51 percent of respondents had "no objection to a CORONA-app, under strict conditions."[38] As legal researcher Domenico Orlando points out, many contact tracing applications like Italy's Immuni app, however, require at least 60 percent of the population to participate for it to be effective.[39] He adds that while whether or not "trust and social acceptance will follow is an unknown variable" when it comes to digital contact tracing, having a strong set of legal protections in place—such as those found with E.U. data protection law and the efforts of the European Data Protection Board (EDPB) to create data protection guidelines for digital surveillance tools[40]—at least makes the technology more feasible.[39] Political and cultural context also plays a role in social acceptance. Countries like South Korea and states like Taiwan—in comparison to mainland China—arguably have seen greater adoption of digital surveillance due to a more relaxed political context[41], though attempts by the citizenry to be vigilantes who shame and discriminate others based on contact tracing data may work counter to that overall adoption rate.[35] Recent past experiences with digitally tracing MERS and SARS outbreaks in both cases, however, have probably played an even greater role in the success of convincing those governments' citizenry to voluntarily participate.[37] Similar adoption success can be seen in a Singapore poll that showed 70 percent of respondents were open to installing its government's voluntary TraceTogether app, which is based on the privacy-preserving protocol BlueTrace.[37]

Challenges in the U.S.

In the United States, adopted use of contract tracing apps has been much more sporadic. Several cultural factors contribute. First and foremost, Americans' reluctance to answer calls from unknown numbers is continuing to increase. A First Orion 2019 Spam Call Trends and Projections Report found that more than 40 percent of calls received are spam calls, and 70 percent of Americans do not answer calls from an unknown number.[42] A December 2019 TrueCaller report found that robocalls in the U.S. "increased from 7% [in 2018] to a staggering 35% – which means that more than every third spam call a user gets is a robocall."[43] This issues get complicated even further by lack of enforcement on "spoofing," the practice of making a phone number appear to originate locally, or from a specific business. Even if a call looks legitimate, it may actually not be.[42][44] As Vice writer Casey Johnston suggests, given spam and spoofing practices and the related lack of enforcement, "until the United States takes its scam and spam call problem seriously, we’re probably going to be in the contact-tracing dark ages for a very long time."[44]

Second, overall trust in the federal government to do what's best and resolve problems has some catching up to do. September 2020 polls by the Pew Research Center[45] and Gallup[46] showed faith in overall federal government near all-time lows. As Lydia Saad of Gallup concluded in her 2020 analysis: "As the country is engaged in critical efforts to combat the medical, economic and societal effects of the global coronavirus pandemic, Americans' trust in the federal government to handle domestic issues is near its lowest point in Gallup trends since 1972, as is their trust in the executive and legislative branches, public officials generally, and the American people themselves."[46] A few positive signs have shown, with overall public trust in government making a small uptick and "trust in the federal government to provide accurate information about COVID-19" rising with the January 2021 presidency change.[47][48] Yet as long as overall trust in the federal government remains low, it will continue to be difficult to roll out any significant contract tracing efforts across the nation.

Third, the ebbs and flows of case numbers—with surges in November 2020 and again in the summer of 2021 with the delta variant[49][50]—make the number of cases to track tough to overcome. This is often compounded by the time it takes to get results, often too long to stem the flow of new infections.[51] A lack of a clear and organized national strategy for contract tracing, unlike the European Union[52], punctuates the challenges of getting more people to use contact tracing apps.[53]

Privacy and other issues

The preservation of user privacy and sensitive identifying data are also vital considerations in digital tracing apps. The previously mentioned BlueTrace protocol and its open-source reference implementation OpenTrace strive to address most of those concerns when "logging Bluetooth encounters between participating devices to facilitate contact tracing, while protecting the users’ personal data and privacy." More specifically, it strives to limit collection of personally-identifiable information, locally store and lock down encounter history, prevent third-party tracking, and provide revocable consent to store and allow use of encounter data.[54] Ultimately, these and other such solutions must ensure the anonymity of tracing data doesn't become compromised. Lew and Anderson also note that "[c]loud-based storage of anonymous identifier beacons may also threaten security, given that any centralized list of identifiers could theoretically be hacked and re-identified."[36] As noted previously, re-identified data, let alone leaked anonymized data, could lead to shaming and discrimination against individuals identified as being infected.[35]

Privacy is not solely a concern of application developers, however. Government entities should also be held responsible for better ensuring how informatics solutions are created and used for epidemiological tracking. The European Union's EDPB and its contact tracing guidelines, adopted in April 2020, offer a compromise between the societal needs of contact tracing and the individual needs of privacy. The EDPB addresses the topics of location data sources and anonymized use, as well as the recommendations and functional requirements of applications employing those data sources, concluding "that one should not have to choose between an efficient response to the current crisis and the protection of our fundamental rights: we can achieve both."[40] The U.S. CDC is less eloquent with its COVID-19 digital contact tracing guidelines, though they still stress the preference for secure data transfer mechanisms, open-source architecture, need-to-know-only access to data for public health authorities, and the ability for users to revoke data access consent at any time.[32]

Finally, most digital tracing solutions also suffer from a few additional downsides. Several researchers have expressed concerns about data accuracy and actionability issues in mobile devices that lead to false positives. These issues include[36][54]:

  • inaccurate inter-device distance detection
  • inaccurate exposure detection in high-density buildings with multiple walls
  • inability to track duration of exposure (i.e., seconds or hours)
  • failures with the underlying technology itself, including BlueTooth or the operating system

The future

Despite these and other drawbacks, various governments around the world have found greater success at managing the COVID-19 pandemic with digital contact tracing. Huang et al. in the Harvard Business Review wonder if the successes reported in South Korea, Taiwan, and other parts of the East could be replicated in the United States and other Western democracies[37]:

Can Western democracies achieve the results seen in East Asia without emulating their means? Probably not. There is likely a fundamental conflict between these requirements and deeply entrenched Western liberal values, such as the expectation of privacy, consent, and the sanctity of individual rights ... At the time of publication, at least three local governments in the United States are considering adoption of a contact-tracing app developed in a project led by MIT, Reuters reports ... But for such technologies to be effective, compliance must be nearly universal. Without a government mandate in the U.S., it’s hard to imagine universal voluntary adoption of even a privacy-protecting tracing app.
 
Maybe Covid-19 is a sign of our future steady state. Different societies will make different choices about how to respond to the next pandemic. For Western democracies the time has come to either rethink our values around the tradeoff between personal privacy and public safety in a pandemic or to accelerate technology innovation and policy development that can preserve both.

  1. 1.0 1.1 El-Kareh, R.; Hasan, O.; Schiff, G.D. (2013). "Use of health information technology to reduce diagnostic errors". BMJ Quality & Safety 22 (Suppl. 2): ii40–ii51. doi:10.1136/bmjqs-2013-001884. PMC PMC3786650. PMID 23852973. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3786650. 
  2. 2.0 2.1 2.2 2.3 National Academies of Sciences, Engineering, and Medicine (2015). "Chapter 5: Technology and Tools in the Diagnostic Process". Improving Diagnosis in Health Care. The National Academies Press. pp. 217–62. doi:10.17226/21794. ISBN 9780309377720. https://www.nap.edu/read/21794/chapter/7. 
  3. 3.0 3.1 Winters-Miner, L.A.; Bolding, P.S.; Hilbe, J.M. et al. (2015). "Chapter 3: Biomedical Informatics". Practical Predictive Analytics and Decisioning Systems for Medicine. Academic Press. pp. 42–59. doi:10.1016/B978-0-12-411643-6.00003-X. ISBN 9780124116436. 
  4. Mandl, K.D.; Overhage, J.M.; Wagner, M.M. et al. (2004). "Implementing syndromic surveillance: A practical guide informed by the early experience". JAMIA 11 (2): 141–50. doi:10.1197/jamia.M1356. PMC PMC353021. PMID 14633933. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC353021. 
  5. Jones, R.G.; Johnson, O.A.; Baststone, G. (2014). "Informatics and the Clinical Laboratory". The Clinical Biochemist Reviews 35 (3): 177–192. PMC PMC4204239. PMID 25336763. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4204239. 
  6. Raeen, M.R. (2018). "How laboratory informatics has impacted healthcare overall". Applied Research Projects 54. doi:10.21007/chp.hiim.0056. https://dc.uthsc.edu/hiimappliedresearch/54. 
  7. Chawla, R.; Goswami, B.; Singh, B. et al. (2010). "Evaluating laboratory performance with quality indicators". Laboratory Medicine 41 (5): 297–300. doi:10.1309/LMS2CBXBA6Y0OWMG. 
  8. Price, C.P. (2001). "Poing of care testing". BMJ 322 (7297): 1285–8. doi:10.1136/bmj.322.7297.1285. PMC PMC1120384. PMID 11375233. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1120384. 
  9. Ziaugra, K.; Hawrylak, V.; Bickley, T. et al. (20 March 2019). "Using analytics to manage QA and reduce laboratory errors". Medical Laboratory Observer. https://www.mlo-online.com/information-technology/lis/article/13017560/using-analytics-to-manage-qa-and-reduce-laboratory-errors. Retrieved 25 April 2020. 
  10. Gundler, D. (23 January 2019). "POCT made easier with informatics". Medical Laboratory Observer. https://www.mlo-online.com/home/article/13017228/poct-made-easier-with-informatics. Retrieved 25 April 2020. 
  11. Healthcare Information and Management Systems Society (2020). "Interoperability in Healthcare". Healthcare Information and Management Systems. https://www.himss.org/resources/interoperability-healthcare. Retrieved 28 April 2020. 
  12. Kun, L.; Coatrieux, G.; Quantin, C. et al. (2008). "Improving outcomes with interoperable EHRs and secure global health information infrastructure". Studies in Health Technology and Informatics 137: 68–79. PMID 18560070. 
  13. Global Center for Health Innovation (28 March 2024). "Improving Patient Care through Interoperability" (PDF). Global Center for Health Innovation. http://s3.amazonaws.com/rdcms-himss/files/production/public/Improving-Patient-Carethrough-Interoperability.pdf. Retrieved 17 September 2021. 
  14. Maxmen, A. (2020). "Thousands of coronavirus tests are going unused in US labs". Nature 580 (7803): 312–13. doi:10.1038/d41586-020-01068-3. PMID 32273619. 
  15. John3504 (7 December 2011). "HL7 Interface cost and maintenance". Spiceworks. https://community.spiceworks.com/topic/175107-hl7-interface-cost-and-maintenance. Retrieved 25 April 2020. 
  16. Ausiello, D.; Shaw, S. (2014). "Quantitative Human Phenotyping: The Next Frontier in Medicine". Transactions of the American Clinical and Climatological Association 125: 219–26. PMC PMC4112685. PMID 25125736. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4112685. 
  17. Edwards, R.R.; Dworkin, R.H.; Turk, D.C. et al. (2016). "Patient phenotyping in clinical trials of chronic pain treatments: IMMPACT recommendations". Pain 157 (9): 1851–71. doi:10.1097/j.pain.0000000000000602. PMC PMC5965275. PMID 27152687. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5965275. 
  18. Mousavizadeh, L.; Ghasemi, S. (2020). "Genotype and phenotype of COVID-19: Their roles in pathogenesis". Journal of Microbiology, Immunology, and Infection: 30082-7. doi:10.1016/j.jmii.2020.03.022. PMC PMC7138183. PMID 32265180. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7138183. 
  19. Gattinoni, L.; Chiumello, D.; Caironi, P. (2020). "COVID-19 pneumonia: Different respiratory treatments for different phenotypes?". Intensive Care Medicine. doi:10.1007/s00134-020-06033-2. PMC PMC7154064. PMID 32291463. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7154064. 
  20. 20.0 20.1 Zhang, X.A.; Yates, A.; Vasilevsky, N. et al. (2019). "Semantic integration of clinical laboratory tests from electronic health records for deep phenotyping and biomarker discovery". npj Digital Medicine 2: 32. doi:10.1038/s41746-019-0110-4. PMC PMC6527418. PMID 31119199. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6527418. 
  21. American Association for Clinical Chemistry (21 February 2019). "Strengthening the Chain of Interoperability". CLN Stat. https://www.aacc.org/cln/cln-stat/2019/february/21/strengthening-the-chain-of-interoperability. Retrieved 17 September 2021. 
  22. Coughlin, J.. "COVID-19 Novel Coronavirus Pandemic". ISA - Vocabulary/Code Set/Terminology. Office of the National Coordinator. Archived from the original on 10 November 2020. https://web.archive.org/web/20201110225342/https://www.healthit.gov/isa/covid-19. Retrieved 17 September 2021. 
  23. 23.0 23.1 Centers for Disease Control and Prevention (3 December 2020). "Case Investigation and Contact Tracing : Part of a Multipronged Approach to Fight the COVID-19 Pandemic". Centers for Disease Control and Prevention. https://www.cdc.gov/coronavirus/2019-ncov/php/principles-contact-tracing.html. Retrieved 17 September 2021. 
  24. Rosen, G. (2015). A History of Public Health (Revised Expanded ed.). Johns Hopkins University Press. pp. 46–47. ISBN 9781421416021. https://books.google.com/books?id=q5yeBgAAQBAJ&pg=PA46. 
  25. Mooney, G. (2015). "Chapter Four: Combustible Material - Classrooms, Contact Tracing, and Following-Up". Intrusive Interventions: Public Health, Domestic Space, and Infectious Disease Surveillance in England, 1840–1914. University of Rochester Press. pp. 93–120. ISBN 9781580465274. https://books.google.com/books?id=P1W3CgAAQBAJ&pg=PA93. 
  26. Davidson, R. (1996). "‘Searching for Mary, Glasgow’: Contact Tracing for Sexually Transmitted Diseases in Twentieth-Century Scotland". Social History of Medicine 9 (2): 195–214. doi:10.1093/shm/9.2.195. 
  27. Wigfield, A.S. (1972). "27 Years of Uninterrupted Contact Tracing: The 'Tyneside Scheme'". British Journal of Venereal Diseases 48 (1): 37–50. doi:10.1136/sti.48.1.37. PMC PMC1048270. PMID 5067063. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1048270. 
  28. 28.0 28.1 Lim, W.T.L. (2006). "Development of Medical Informatics in Singapore - Keeping Pace with Healthcare Challenges" (PDF). Proceedings from the 2006 Meeting of the Asia Pacific Association for Medical Informatics: 1–4. https://www.apami.org/apami2006/papers/wlim(sg).pdf. 
  29. Zhang, Y.; Dang, Y.; Chen, Y.-D. et al. (2008). "BioPortal Infectious Disease Informatics research: Disease surveillance and situational awareness". Proceedings of the 2008 International Conference on Digital Government Research: 393–94. doi:10.5555/1367832.1367909. 
  30. Schafer, I.J.; Knudsen, E.; McNamara, L.A. et al. (2016). "The Epi Info Viral Hemorrhagic Fever (VHF) Application: A Resource for Outbreak Data Management and Contact Tracing in the 2014–2016 West Africa Ebola Epidemic". The Journal of Infectious Diseases 214 (Suppl. 3): S122–S136. doi:10.1093/infdis/jiw272. PMID 27587635. 
  31. 31.0 31.1 Ha, Y.P.; Tesfalul, M.A.; Littman-Quinn, R. et al. (2016). "Evaluation of a Mobile Health Approach to Tuberculosis Contact Tracing in Botswana". Journal of Health Communication 21 (10): 1115-21. doi:10.1080/10810730.2016.1222035. PMC PMC6238947. PMID 27668973. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6238947. 
  32. 32.0 32.1 American Hospital Association (4 May 2020). "CDC sets preliminary standards for digital COVID-19 contact tracing tools". American Hospital Association. https://www.aha.org/news/headline/2020-05-04-cdc-sets-preliminary-standards-digital-covid-19-contact-tracing-tools. Retrieved 19 May 2020. 
  33. Yan, H. (15 May 2020). "Contact tracing 101: How it works, who could get hired, and why it's so critical in fighting coronavirus now". CNN Health. https://www.cnn.com/2020/04/27/health/contact-tracing-explainer-coronavirus/index.html. Retrieved 19 May 2020. 
  34. Fortin, J. (18 May 2020). "So You Want to Be a Contact Tracer?". The New York Times. https://www.nytimes.com/2020/05/18/health/coronavirus-contact-tracing-jobs.html. Retrieved 19 May 2020. 
  35. 35.0 35.1 35.2 35.3 Waltz, E. (25 March 2020). "Halting COVID-19: The Benefits and Risks of Digital Contact Tracing". IEEE Spectrum. https://spectrum.ieee.org/halting-covid19-benefits-risks-digital-contact-tracing. Retrieved 17 September 2021. 
  36. 36.0 36.1 36.2 Lew, C.; Anderson, F. (14 April 2020). "‘Digital Contact Tracing’ — Advantages, Risks, & Post-COVID Applications". DeciBio. https://www.decibio.com/insights/digital-contact-tracing-advantages-risks-post-covid-applications. Retrieved 17 September 2021. 
  37. 37.0 37.1 37.2 37.3 Huang, Y.; Sun, M.; Sui, Y. (15 April 2020). "How Digital Contact Tracing Slowed Covid-19 in East Asia". Harvard Business Review. https://hbr.org/2020/04/how-digital-contact-tracing-slowed-covid-19-in-east-asia. Retrieved 19 May 2020. 
  38. Knowledge Centre Data & Society (8 April 2020). "Survey: 51% has no objection to a corona-app, under strict conditions". Knowledge Centre Data & Society. https://data-en-maatschappij.ai/en/news/survey-corona-app. Retrieved 17 September 2021. 
  39. 39.0 39.1 "The EDPB Guidelines on digital tools against Covid-19, a brief comment". KU Leuven Centre for IT & IP Law. 12 May 2020. https://www.law.kuleuven.be/citip/blog/the-edpb-guidelines-on-digital-tools-against-covid-19-a-brief-comment/. Retrieved 19 May 2020. 
  40. 40.0 40.1 European Data Protection Board (April 2020). "Guidelines 04/2020 on the use of location data and contact tracing tools in the context of the COVID-19 outbreak". European Data Protection Board. https://edpb.europa.eu/our-work-tools/our-documents/guidelines/guidelines-042020-use-location-data-and-contact-tracing_en. Retrieved 17 September 2021. 
  41. Nazeer, T. (7 April 2020). "Digital Surveillance and 'Technological Totalitarianism'". Byline Times. https://bylinetimes.com/2020/04/07/the-coronavirus-crisis-digital-surveillance-and-technological-totalitarianism/. Retrieved 15 May 2020. 
  42. 42.0 42.1 First Orion (2019). "Spam Call Trends and Projections Report - Summer 2019" (PDF). First Orion. http://firstorion.com/wp-content/uploads/2019/07/First-Orion-Scam-Trends-Report_Summer-2019.pdf. Retrieved 21 November 2020. 
  43. Kok, K.F. (3 December 2019). "TRUECALLER INSIGHTS: TOP 20 COUNTRIES AFFECTED BY SPAM CALLS & SMS IN 2019". TrueCaller. https://truecaller.blog/2019/12/03/truecaller-insights-top-20-countries-affected-by-spam-calls-sms-in-2019/. Retrieved 21 November 2020. 
  44. 44.0 44.1 Johnston, C. (31 August 2020). "The Real Reason Contact Tracing Is Doomed in the US: Spam Calls". Vice. https://www.vice.com/en/article/935vvz/the-real-reason-contact-tracing-is-doomed-in-the-us-spam-calls. Retrieved 21 November 2020. 
  45. Pew Research Center (14 September 2020). "Americans’ Views of Government: Low Trust, but Some Positive Performance Ratings". https://www.pewresearch.org/politics/2020/09/14/americans-views-of-government-low-trust-but-some-positive-performance-ratings/. Retrieved 21 November 2020. 
  46. 46.0 46.1 Saad, L. (29 September 2020). "Trust in Federal Government's Competence Remains Low". Gallup. https://news.gallup.com/poll/321119/trust-federal-government-competence-remains-low.aspx. Retrieved 21 November 2020. 
  47. Pew Research Center (17 May 2021). "Public Trust in Government: 1958-2021". https://www.pewresearch.org/politics/2021/05/17/public-trust-in-government-1958-2021/. Retrieved 17 September 2021. 
  48. Durkee, A. (26 January 2021). "Trust In Government Information On Covid Surges Under Biden, Poll Reports". Forbes. Archived from the original on 26 January 2021. https://web.archive.org/web/20210126160459/https://www.forbes.com/sites/alisondurkee/2021/01/26/trust-in-government-information-on-covid-19-surges-under-biden-poll-reports/. Retrieved 17 September 2021. 
  49. Stone, W. (24 November 2021). "How Do We Stop This Surge? Here's What Experts Say Could Help". NPR Shots. https://www.npr.org/sections/health-shots/2020/11/24/937178668/are-more-lockdowns-inevitable-or-can-other-measures-stop-the-surge. Retrieved 17 September 2021. 
  50. Doheny, K. (31 August 2021). "As Delta Surges, Contact Tracing Re-Takes COVID Center Stage". WebMD Health News. https://www.webmd.com/lung/news/20210831/delta-surge-contact-tracing. Retrieved 17 September 2021. 
  51. Khazan, O. (31 August 2020). "The Most American COVID-19 Failure Yet". The Atlantic. https://www.theatlantic.com/politics/archive/2020/08/contact-tracing-hr-6666-working-us/615637/. 
  52. European Commission (19 October 2020). "Coronavirus: EU interoperability gateway for contact tracing and warning apps – Questions and Answers". European Union. https://ec.europa.eu/commission/presscorner/detail/en/QANDA_20_1905. Retrieved 21 November 2020. 
  53. Uberti, D. (21 October 2020). "Disjointed Covid-19 Apps Across U.S. Raise Questions About Tech’s Role". The Wall Street Journal. https://www.wsj.com/articles/disjointed-covid-19-apps-across-u-s-raise-questions-about-techs-role-11603272613. Retrieved 21 November 2020. 
  54. 54.0 54.1 Bay, J.; Kek, J.; Tan, A. et al. (2020). BlueTrace: A privacy-preserving protocol for community-driven contact tracing across borders. pp. 1–9. https://bluetrace.io/static/bluetrace_whitepaper-938063656596c104632def383eb33b3c.pdf.