Journal:Cyberbiosecurity: A new perspective on protecting U.S. food and agricultural system

From LIMSWiki
Jump to: navigation, search
Full article title Cyberbiosecurity: A new perspective on protecting U.S. food and agricultural system
Journal Frontiers in Bioengineering and Biotechnology
Author(s) Duncan, Susan E.; Reinhard, Robert; Williams, Robert C.; Ramsey, Ford; Thomason, Wade;
Lee, Kiho; Dudek, Nancy; Mostaghimi, Saied; Colbert, Edward; Murch, Randall
Author affiliation(s) Virginia Tech, Tyson Foods
Primary contact Email: duncans at vt dot edu
Editors Morse, Stephen Allen
Year published 2019
Volume and issue 7
Page(s) 63
DOI 10.3389/fbioe.2019.00063
ISSN 2296-4185
Distribution license Creative Commons Attribution 4.0 International
Website https://www.frontiersin.org/articles/10.3389/fbioe.2019.00063/full
Download https://www.frontiersin.org/articles/10.3389/fbioe.2019.00063/pdf (PDF)

Abstract

Our national data and infrastructure security issues affecting the “bioeconomy” are evolving rapidly. Simultaneously, the conversation about cybersecurity of the U.S. food and agricultural system (cyber biosecurity) is incomplete and disjointed. The food and agricultural production sectors influence over 20% of the nation's economy ($6.7T) and 15% of U.S. employment (43.3M jobs). The food and agricultural sectors are immensely diverse, and they require advanced technologies and efficiencies that rely on computer technologies, big data, cloud-based data storage, and internet accessibility. There is a critical need to safeguard the cyber biosecurity of our bioeconomy, but currently protections are minimal and do not broadly exist across the food and agricultural system. Using the food safety management Hazard Analysis Critical Control Point (HACCP) system concept as an introductory point of reference, we identify important features in broad food and agricultural production and food systems: dairy, food animals, row crops, fruits and vegetables, and environmental resources (water). This analysis explores the relevant concepts of cyber biosecurity from food production to the end product user (such as the consumer) and considers the integration of diverse transportation, supplier, and retailer networks. We describe common challenges and unique barriers across these systems and recommend solutions to advance the role of cyber biosecurity in the food and agricultural sectors.

Keywords: plant, animal, food, cyber biosecurity, biosecurity, cybersecurity, agriculture, bioeconomy

Introduction: Food and agriculture cyberbiosecurity at the interface of biosecurity and cybersecurity

Public trust and confidence in the food supply are critical and influential on acceptance of data-driven innovations and technologies within the food and agriculture systems. Cyberbiosecurity is a nascent paradigm and discipline at the interface of biosafety/biosecurity, cybersecurity, and cyber-physical security (Figure 1).[1] This new discipline has emerged alongside “big data” with the extensive and ever-increasing reliance of the life sciences on information systems technologies, rapid and profitable expansion of life science discoveries, and the growth of the U.S. bioeconomy. Protecting biological data and information within the life sciences has unique differences from the more familiar biosafety and biosecurity approaches.[2] While the latter two categories address biological risks and threats, they do not protect against harm created when computational and information technology-dependent systems are threatened or corrupted. Just as food safety regulations target the protection of human health, incorporating cyber biosecurity strategies for the food and agriculture industries is a protective step in securing the food supply. Such efforts have the power to positively influence lives and protect the bioeconomy. Cyberbiosecurity can improve the security and stability of domestic and global food and agriculture systems. United States innovation in this realm is routinely studied and adopted around the globe, and as such, the U.S. can provide insight and leadership in cyber biosecurity of global food and agriculture systems.


Fig1 Duncan FrontBioengBiotech2019 7.jpg

Fig. 1 Cyberbiosecurity is an emerging discipline for protecting life sciences data, functions and operations (or infrastructure), and the bioeconomy

Integrated scientific, mathematical, computational, and engineering advancements in regenerative biology, genetics and breeding technologies, plant-derived vaccine and animal therapies, biological design and testing automation, and other activities are rapidly leading to development of biotechnological and agricultural applications of direct relevance to the food and agriculture system.[3][4] The translation and application of data-driven technologies for precision agriculture, autonomous systems, bio-automated processing and data recording, and other technologies yields large data sets of economic and bio-based information for agribusinesses.[5] Such advances require high throughput processing, data management and integration, bio-automation, and other computer-based management of biological data. These advances increase efficiencies, decision processes, and output within the food and agricultural system. However, such information is susceptible to ownership policy challenges, theft, and cyberattack as users may not be alert to potential vulnerabilities nor be trained in effective protections and security strategies.[5][6] Unprotected or weakly protected systems are susceptible to unwanted surveillance, intrusions into data systems, and cyber-activities targeted toward malicious attack. Cyberbiosecurity threats include inappropriate access to systems, data, or analytical technologies and the use or corruption of the information accessed to cause harm within life science-focused research, production, processing, and use. Examples of data-driven, high-value food and agricultural products susceptible to cyber threat include high-yielding and specialty agricultural crops, high performance livestock, biopharma fermented molecules developed through advanced breeding and genomics, biotechnology advancements, and “big data” analyses.[7] As technology advances, all parts of society, from governmental agencies to public health and manufacturing, rely more on advanced biological systems with big data and technologies that utilize such information. The identification and mitigation of cyber biosecurity threats will become increasingly important.

Vulnerability of the food and agricultural system and the bioeconomy

The U.S. food and agriculture system, influencing 20% ($6.7T) of the domestic bioeconomy[8], represents a significant risk to global food security. The data science market value for agriculture is estimated in excess of $20B.[5] The food and agriculture system is composed of many sectors that are not well-integrated, is widely dispersed geographically, and has huge diversity in size (number of employees) and capacity. Most of the economic value is generated by large, multinational corporate enterprises. Conversely, small family-owned farming operations account for 90% of U.S. farms, which yield 24% of the value of agricultural production.[9]

The family small-business agricultural enterprise (family farm) has economic and social distinctions from corporate farms. Small farm producers view their data with a sense of personal privacy and protection.[5] Small businesses often use their internet-linked home computer for both personal and business activities, increasing the risk of cyber-attack[10][11]; over 20% of small businesses get hacked.[11] Generally, small farms and agribusinesses are not comfortable adopting computer security technology (selecting, configuring, managing) although they recognize its relevance and value. Moderate-sized agribusinesses, including many food processing companies and supporting industries, are vulnerable since cyberattacks are often targeted against organizations with more than 100 employees.[11] Additionally, the food and agriculture system includes military food production—such as the manufacturing of packaged meals for soldiers—which has a high potential for sabotage.[12] It is important to note that attackers need not know details of the food manufacturing process. Attackers need only know technical methods for exploiting the machinery or the process, such as lowering the temperature on meat cookers before packaging.[13][14]

The incorporation of cyber-based technologies and data driven solutions in farm production, food processing, supplier industries, transport of goods, regulatory oversight, and marketing sales and communication with consumers creates a paradigm shift.[6] Cloud-based storage of large data sets, use of open-sourced or internet/cloud-based software, and corporate management of proprietary software each increase opportunities for data access by unauthorized users. Within the food and agriculture system, the use of biological and genetic analytical technologies within research laboratories is widespread for the evaluation of food quality, identification of zoonotic disease, and animal and plant health. Additionally, the use of bioinformatics and genetic technologies is enhancing the rate of development of new products and crops. Public trust and acceptance are key to incorporating advanced technologies into the food and agriculture system.[4][15] Interdependency of information technology with biological output creates opportunities for new bio-threats, which can harm public trust; transparency is valued.[7] When public opinion is turned against a technical advancement, policy and protection strategies may cause more harm than the actual threat itself.[4]

Holistically, the ramifications of a failure to provide cyber biosecurity of the food and agriculture system fall into several general categories[6]:

  • Threats to confidentiality—data privacy
    • Data exposure (e.g., naïve exposure of data by individuals, cybersecurity gaps in small businesses, or laboratories to potential threats)
    • Capturing private data with intent to aggregate data for profit or predictive advantage
  • Threats to integrity—theft or destruction of intellectual property/productivity disruptions, and safety risks
    • Intellectual property theft (e.g., advances in plant and animal varieties and genetics)
    • Manipulation of critical automated (computer-based) processes (e.g., thermal processing time and temperature for food safety)
    • Seizing control of robotics or autonomous vehicles (e.g., failure to perform, overriding of precise function)
  • Threats to availability—disruption of agricultural/food production and supply
  • Misinformation influencing trust and cooperation within the food and agriculture system and/or consumers
  • Lack of equipment, supplies, or end-products to meet expectations
  • Lack of ability to perform vulnerability assessments and develop emergency response plans (e.g., protection of rivers, surface waters, and drinking water supplies)

The food and agricultural industries are at a critical point as the development and use of biological, genetic, precision, and information technologies expand and intersect. Collectively, there is a need to evaluate potential liabilities and understand the vulnerabilities of biological and genetic data systems.

Risk assessment, critical control points, and regulatory options

Cybersecurity risk assessment for industrial control systems (ICS) is advancing rapidly. Cherdantseva et al.[16] reviewed 24 different cybersecurity risk assessment methods relevant to ICS. Applications of such risk assessment approaches in the food and agriculture sectors have not been evaluated, and the complexity and diversity of those sectors may not conform to the current cybersecurity risk assessment methods. Cyberbiosecurity risk assessment strategies that address the unique security challenges at the intersection of the biological, physical, and cyberspace are important for protecting the food and agriculture system.

Food manufacturers use the principles of Hazard Analysis and Critical Control Points (HACCP) to assure the production of safe products. HACCP is a familiar risk assessment process within the food and agriculture system. This management system looks at the likely occurrence of a chemical, biological, or physical food safety hazard in the manufacturing process and the controls that can be put in place to reduce, eliminate, or control the potential hazard. HACCP principles use critical control points (CCPs) as steps in a process where specific controls can be implemented to control, reduce, or eliminate a hazard. HACCP principles are used around the world for the production of safe food products and are required by the U.S. Department of Agriculture's Food Safety Inspection Service and the U.S. Food and Drug Administration. A risk matrix (see Table 1 in the Supplemental Material) may be used to identify potential vulnerabilities and estimate likelihood of occurrence with the potential public health and financial consequences. An example using HACCP principles for an assessment of an industrial laboratory processing biological and genetic materials is presented in the Supplemental Material. In this specific example, two CCPs (alternative supplier verification of biological and genetic materials program, and cyber biosecurity data verification program) were identified to mitigate potential risks. Four control point programs (supplier approval, employee training, security programs, and good laboratory standard operating procedures) were identified to support the overarching process for cyber biosecurity.

Several economic problems confront policymakers when addressing cyber biosecurity in the food and agriculture sector. The most pressing concerns are externalities caused by the networked nature of the system and the misaligned incentives of individual agents. The risks associated with cyber biosecurity threats and harm to society are likely to be larger than the losses suffered by an individual entity; individual firms may not have incentives to provide socially optimal levels of security for the network. Furthermore, if agents know that their own protection depends on security investments made by others, they may become free-riders. Again, this results in inadequate private provision of the public good or security of the network.[17]

Multiple regulatory and policy options exist to counter threats to the food and agriculture system. In some cases, it may be easier to implement protections within these sectors because agribusinesses are already subject to relatively strict disclosure regulations. Information disclosure provides regulators with the data necessary to align individual incentives with the security of the system as a whole. This could be done with top-down regulation, changes to the assignment of liability, or the development of market-based systems for the control of cyber biosecurity risks. For instance, the development of cyber biosecurity insurance markets could be encouraged. Regardless of eventual policy measures, it will be important to ensure that the costs of protecting the system are properly aligned with the probabilities of loss and magnitudes of loss associated with cyber biosecurity threats. The most efficient methods of securing the food and agriculture system are likely to rely on a variety of regulatory approaches.

Considering the diversity within and across plant, animal, and environmental sectors of the food and agricultural system

The HACCP concept assesses risk and establishes CCPs for a specific facility and cannot be generalized effectively to all food manufacturing plants. Applying this concept to cyber biosecurity risk, control points, and CCPs, therefore, is challenged by the diversity of enterprises within a sector and across the food and agriculture system. Within each sector are unique suppliers providing biological material, chemicals and ingredients, robotics and machinery, software, data, and data storage systems. Some of the security measures are encompassed by cybersecurity, cyberphysical security, and biosecurity/biosafety practices, at least for large corporate entities with sufficient resources. However, an unsecured system from a small agribusiness supplier, producer, processor, or commodity cooperative could introduce risk.

We use the illustration of a train with multiple boxcars as an example of various sectors within one commodity sector of the food and agriculture system (Figure 2, top). The various cars represent the transition from genetics and breeding through production, processing, distribution, and consumer purchase/use. The exchange of information between the different sectors is often limited, as illustrated by the couplings. The role of the federal government policies and programs provide support and guidance (tracks). Suppliers and other support systems access one or more sectors within a commodity system. The system is driven (engine) by general public (consumers) acceptance of practices and goods, or their fear and mistrust if a risk or threat is perceived. If any stage “derails” or if any supporting agency or organization “buckles” due to a cyber biosecurity threat or attack, the entire system is at risk, with subsequent risk to the U.S. food supply and the bioeconomy (Figure 2, bottom). Currently, the cybersecurity industry is not visibly involved in protecting biological data interfacing with the cyber-physical infrastructure supporting the food and agriculture system.


Fig2 Duncan FrontBioengBiotech2019 7.jpg

Fig. 2 (Top) Food and agriculture system for each commodity sector is a sequence of stages, with limited communications and sharing of data between each; (Bottom) if a cyber-biosecurity event occurs, it can have catastrophic effect on the entire food and agriculture system.

Some potential mitigations to the issues are possible. Cyber biosecurity planning and implementation are needed to protect the intellectual and physical (data) property associated with such food and agriculture priorities. Examples include:

  • Plant and animal germplasm, such as old world corn germplasm, microbiology collection (pathogens, fermentation, microbiome) repositories, including economic assessment and protection of data sharing;
  • Biocontrolled systems or processes, such as “smart” technology greenhouse data;
  • Animal and plant disease diagnostic networks and information sharing;
  • Fermentation processing and thermal processing control parameters; and
  • Freshwater and drinking water supplies and treatment systems.

We further illustrate by outlining some unique considerations for various food and agriculture commodities.

Dairy

Selection of genetics for breeding is key to the high milk production in the U.S. dairy industry. Genetic data is highly evaluated as part of the process for breeding. Milk production records are important for establishing high-performance animals. While there are some very large dairy herds (>2,000 animals), the U.S. dairy industry is dominated by small to medium farms, many of whom sell their milk through a cooperative structure. Herd health records and drug use are regulated. Data security is variable, and often limited. Fluid milk and dairy food processors do not have detailed records of individual cow production or farm production practices, creating a gap in tracing of information and potential for data breach. Processors utilize computer systems for maintaining processing temperatures, ingredient additions, sanitizing, and cleaning steps.

Food animals

Selective breeding is critical to maximize genetic gain during food animal production. For instance, multiple lines of breeds are incorporated into swine production to enhance heterogeneity. Pedigree information of the breeds significantly influences selection of founders for the production system. Breach or manipulation of the information can lead to a devastating loss to producers. Recent development in genomic-based selection strategies[18] may also be vulnerable to cyber biosecurity threats as the genomic information can be targeted or exploited. Potential application of genome editing technology in food animals[19] may also generate novel genetic information that could dramatically improve productivity of food animals.

Row crops

Similar to the dairy industry, the row crop sector consists of a large number of farms of varying size. Grain is typically commingled at the first point of sale and often aggregated further during the process of storage and handling, greatly limiting traceability.[20] Modern farms using precision agriculture technologies generate enormous amounts of data, about everything from soil conditions to machinery performance and location; such information is often controlled by agriculture technology providers.[5][6] Securing data and preventing breaches across all these systems is difficult and is frequently an afterthought by the actual users.[21] Individual producer data is often sent directly to a third-party entity for data storage, cleaning, and processing. Many aggregate data and use this as market information or sell it to other companies who do. Commodity traders may use some data streams to guide investment. Anonymization typically occurs at the time of aggregation, but questions exist about the effectiveness of these techniques. After transfer, data security becomes the responsibility of the third-party data management company, but these entities are themselves not immune from security breaches and would be vulnerable to security issues inserted upstream at the farm or machinery level. Finally, commodity markets are strongly influenced by crop production estimates generated by surveys of farmers and the agriculture industry.

Fruits and vegetables

Fresh fruits and vegetables are leading sources for foodborne illness in the United States.[22] Furthermore, even in the absence of foodborne illness outbreaks, fresh produce recalls occur regularly due to the presence of potential harmful microorganisms. Fresh produce available for sale in local markets may have been produced in one of many locations throughout the nation or from one of many countries around the world. The production, sorting, grading, commingling, transporting, marketing, and sale of fresh fruits and vegetables is complex and involves numerous industry actors with varying roles. Tracking fresh produce from initial production through consumption is critical to limit the potential for and impact of foodborne illness outbreaks. Accurate product information and rapid access to data is essential to identify contaminated product in the market, prevent or limit foodborne illness, limit the damage to non-implicated producers, and maintain consumer confidence. Access to product tracking and microbiological data is increasing in the fresh produce industry.

Environmental resources (water)

Drinking water safety is extremely important on-farm, for food processing, ensuring consumers' health and proper functioning of the ecosystem. The proportion of the world's population consuming drinking water from certified and controlled water sources is about 90% and still increasing.[23] However, 2.3 billion people worldwide suffer from diseases related to drinking water. Over the past three decades, significant drinking water contamination incidents have occurred in developing as well as developed countries, creating health problems for consumers.[24][25] Traditional risk management systems, based on addressing and correcting the failure after its occurrence, are inadequate to deal with potential cyber biosecurity threats (as the cybersecurity landscape is changing rapidly as technology continues to advance). Given the severity of risk and potential harm, cyber biosecurity must be given a high priority for the drinking water management and treatment sector.[26]

Conclusions: Moving Toward Solutions

The complex and vastly diverse enterprises within the food and agriculture system increases vulnerability of our food supply and threatens our ability to contribute to the global food supply. Rapid advancements in technologies and adoption into the food and agriculture sectors increase the risks for cyber biosecurity threats and attacks. The current food and agriculture workforce has limited knowledge or training appropriate to evaluate and protect the vast amount of data generated by these technologies. The cybersecurity industry is not well-prepared to address the unique structure and functions within food and agriculture sectors. Protecting them includes (1) developing and characterizing effective cyber biosecurity risk assessment and mitigation strategies; (2) developing and preparing the current and future workforce to identify, address, and adopt effective cyber biosecurity strategies; (3) considering policy and regulations, including insurance, for protection within and across the entire system; and (4) effectively communicating within the sector and across the food and agriculture system.[15] Awareness, knowledge, adoption, and frequent evaluation of cyber biosecurity plans and strategies among and within all sectors is essential. A multidisciplinary approach integrating expertise in agriculture, food, engineering, computer science, and cybersecurity is needed for filling this gap. The USDA, in consultation with academic, public, and private sector experts and representation from sectors within the food and agriculture system, should lead an initiative for developing a planned approach to addressing cyber biosecurity. Private and public funding is needed to support research priorities and implementation strategies. Checkoff funding mechanisms or cooperative agreements, which are common within commodity systems, may be options for assisting small to moderate-sized agribusinesses. Workforce development, effective communication strategies, and cooperation across sectors and industries will help increase support and compliance, reducing the risks and providing increased protection for the U.S. bioeconomy and our domestic and global food supply.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fbioe.2019.00063/full#supplementary-material

Acknowledgements

The authors acknowledge the Virginia Agricultural Experiment Station, Blacksburg, VA for financial support for publishing this manuscript. The authors gratefully acknowledge Michael J. Stamper, Data Visualization Designer and lecturer at the University Libraries, Data Services at Virginia Tech, for his contributions to the figures.

Author Contributions

SD lead author, responsible for structure, content, and figure; responsible for considering, incorporation co-author contributions and suggested edits; responsible for final version. RR provided draft content related to HACCP and post-harvest processing and cyber biosecurity; contributed, reviewed, and edited the manuscript. RW contributed, reviewed, and edited content related to HACCP and post-harvest processing and biosecurity; reviewed and critiqued manuscript to ensure quality and flow. FR contributed, reviewed, and edited content related to food and agriculture system influence on bioeconomy; reviewed and critiqued manuscript to ensure quality and flow. WT contributed, reviewed, and edited content related to cyber biosecurity in agriculture (pre-harvest; crop, soil, and environment); reviewed and critiqued manuscript to ensure quality and flow. KL contributed, reviewed, and edited content related to cyber biosecurity in agriculture (pre-harvest; animal breeding and genetics); reviewed and critiqued manuscript to ensure quality and flow. ND contributions to sections relating to biotechnology; overall quality assurance and readability reviews and modifications. SM contributed, reviewed, and edited content related to cyber biosecurity in food and agricultural system and the environment; reviewed and critiqued manuscript to ensure quality and flow. EC contributed, reviewed, and edited content related to cybersecurity, data sources, and integration into the food and agricultural system; reviewed and critiqued manuscript to ensure quality and flow. RM co-originator of the cyber biosecurity concept; co-originator of the concepts relating to food and agricultural system; contributed, reviewed, and edited content related to cyber biosecurity, data sources, and integration into the food and agricultural system; reviewed and critiqued manuscript to ensure quality, flow, and relevance to the targeted audience.

Conflict of interest

RR is employed by Tyson Foods.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

References

  1. Murch, R.S.; So, W.K.; Buchholz, W.G. et al. (2018). "Cyberbiosecurity: An Emerging New Discipline to Help Safeguard the Bioeconomy". Frontiers in Bioengineering and Biotechnology 6: 39. doi:10.3389/fbioe.2018.00039. 
  2. Peccoud, J.; Gallegos, J.E.; Murch, R. et al. (2018). "Cyberbiosecurity: From Naive Trust to Risk Awareness". Trends in Biotechnology 36 (1): 4–7. doi:10.1016/j.tibtech.2017.10.012. PMID 29224719. 
  3. Board on Chemical Sciences and Technology; Board on Life Sciences (2014). Meeting Recap: Workshop - Convergence: Safeguarding Technology in the Bioeconomy. The National Academies of Sciences, Engineering, and Medicine. 
  4. 4.0 4.1 4.2 Wintle, B.C.; Boehm, C.R.; Rhodes, C. et al. (2017). "A transatlantic perspective on 20 emerging issues in biological engineering". eLife 3: e30247. doi:10.7554/eLife.30247. PMC PMC5685469. PMID 29132504. http://www.pubmedcentral.nih.gov/articlerender.fcgi?tool=pmcentrez&artid=PMC5685469. 
  5. 5.0 5.1 5.2 5.3 5.4 Sykuta, M.E. (2016). "Big data in agriculture: Property rights, privacy and competition in ag data services". International Food and Agribusiness Management Review 19 (A): 57–74. https://www.ifama.org/Volume-19-Issue-A. 
  6. 6.0 6.1 6.2 6.3 Boghossian, A.; Linsky, S.; Brown, A. et al. (2018). "Threats to Precision Agriculture" (PDF). U.S. Department of Homeland Security. https://www.dhs.gov/sites/default/files/publications/2018%20AEP_Threats_to_Precision_Agriculture.pdf. Retrieved 08 January 2019. 
  7. 7.0 7.1 Board on Chemical Sciences and Technology (2015) (PDF). Meeting Recap: Safeguarding the Bioeconomy: Applications and Implications of Emerging Science. The National Academies of Sciences, Engineering, and Medicine. https://www.ehidc.org/sites/default/files/resources/files/Safeguarding%20the%20Bioeconomy_II_Recap%20Final%20090815.pdf. 
  8. "What is the Food and Ag Industries’ Impact in Your Community?". Feeding the Economy. 2018. https://feedingtheeconomy.com/. Retrieved 28 October 2018. 
  9. MacDonald, J.M.; Hoppe, R.A. (06 March 2017). "Large Family Farms Continue To Dominate U.S. Agricultural Production". Amber Waves. U.S. Department of Agriculture. https://www.ers.usda.gov/amber-waves/2017/march/large-family-farms-continue-to-dominate-us-agricultural-production/. Retrieved 28 October 2018. 
  10. National Agricultural Statistics Service (20 August 2013). "Farm Computer Usage and Ownership". U.S. Department of Agriculture. https://usda.library.cornell.edu/concern/publications/h128nd689?locale=en. Retrieved 27 October 2018. 
  11. 11.0 11.1 11.2 Geil, A.; Sagers, G.; Spaulding, A.D. et al.. "Cyber security on the farm: An assessment of cyber security practices in the United States agriculture industry". International Food and Agribusiness Management Review 21: 317–34. doi:10.22434/IFAMR2017.0045. 
  12. Colbert, E.J.M.; Kott, A.; Knachel, L.P.. "The game-theoretic model and experimental investigation of cyber wargaming". The Journal of Defense Modeling and Simulation: Applications, Methodology, Technology: 1–18. doi:10.1177/1548512918795061. 
  13. Colbert, E.; Sullivan, D.; Wong, K. et al. (2015). Table-Top Exercise Final Report: Intrusion Detection Capabilities for US Army SCADA Systems: Information Packet. US Army Research Lab Technical Report ARL-TR-7498. 
  14. Colbert, E.; Sullivan, D.; Wong, K. et al. (2015). RED and BLUE Teaming of a US Army SCADA System: Table-Top Exercise Final Report. US Army Research Lab Technical Report ARL-TR-7497. 
  15. 15.0 15.1 National Institute of Food and Agriculture (01 November 2016). "Data Summit: Changing the Face, Place, and Space of Agriculture" (PDF). U.S. Department of Agriculture. https://nifa.usda.gov/sites/default/files/resource/Stakeholder%20Ideas%20Engine%20Input%20-%20Summary%5B1%5D.pdf. Retrieved 28 October 2018. 
  16. Cherdantseva, Y.; Burnap, P.; Blyth, A. et al. (2016). "A review of cybersecurity risk assessment methods for SCADA systems". Computers & Security 56: 1–27. doi:10.1016/j.cose.2015.09.009. 
  17. Varian, H. (2004). "System Reliability and Free Riding". In Camp, L.J.; Lewis, S.. Economics of Information Security. Advances in Information Security. 12. Springer. pp. 1–15. doi:10.1007/1-4020-8090-5_1. ISBN 9781402080906. 
  18. Sellner, E.M.; Kim, J.W.; McClure, M.C. et al. (2007). "Board-invited review: Applications of genomic information in livestock". Journal of Animal Science 85: 12. doi:10.2527/jas.2007-0291. PMID 17709778. 
  19. Telugu, B.P.; Park, K.E.; Park, C.H. (2017). "Genome editing and genetic engineering in livestock for advancing agricultural and biomedical applications". Mammalian Genome 28 (7–8): 338–47. doi:10.1007/s00335-017-9709-4. PMID 28712062. 
  20. Golan, E.; Krissoff, B.; Kuchler, F. et al. (March 2004). "Traceability in the U.S. Food Supply: Economic Theory and Industry Studies". USDA Economic Research Service. https://www.ers.usda.gov/publications/pub-details/?pubid=41632. 
  21. Ferris, J.L. (2017). "Data Privacy and Protection in the Agriculture Industry: Is Federal Regulation Necessary?". Minnesota Journal of Law, Science & Technology 18 (1): 309–42. https://scholarship.law.umn.edu/mjlst/vol18/iss1/6. 
  22. Callejón, R.M.; Rodríguez-Naranjo, M.I.; Ubeda, C. et al. (2015). "Reported foodborne outbreaks due to fresh produce in the United States and European Union: Trends and causes". Foodborne Pathogens and Disease 12 (1): 32–8. doi:10.1089/fpd.2014.1821. PMID 25587926. 
  23. Vierira, J.M. (2011). "A strategic approach for Water Safety Plans implementation in Portugal". Journal of Water and Health 9 (1): 107–16. doi:10.2166/wh.2010.150. PMID 21301119. 
  24. Hamilton, P.D.; Gale, P.; Pollard, S.J. (2006). "A commentary on recent water safety initiatives in the context of water utility risk management". Environment International 32 (8): 958–66. doi:10.1016/j.envint.2006.06.001. PMID 16870255. 
  25. Tsoukalas, D.S.; Tsitsifli, S. (2018). "A Critical Evaluation of Water Safety Plans (WSPs) and HACCP Implementation in Water Utilities". Proceedings 2 (11): 600. doi:10.3390/proceedings2110600. 
  26. Germano, J.H. (2018). "Cybersecurity Risk and Responsibility in the Water Sector" (PDF). American Water Works Association. https://www.awwa.org/Portals/0/AWWA/Government/AWWACybersecurityRiskandResponsibility.pdf?ver=2018-12-05. Retrieved 08 January 2019. 

Notes

This presentation is faithful to the original, with only a few minor changes to presentation, grammar, and punctuation. In some cases important information was missing from the references, and that information was added.