Journal:Automated cyber and privacy risk management toolkit
|Full article title||Automated cyber and privacy risk management toolkit|
|Author(s)||Gonzalez-Granadillo, Gustavo; Menesidou, Sofia A.; Papamartzivanos, Dimitrious; Romeu, Roman; Navarro-Llobet, Diana; Okoh, Caxton; Nifakos, Sokratis; Xenakis, Christos; Panaousis, Emmanouil|
|Author affiliation(s)||Atos Spain, UBITECH Ltd., Fundació Privada Hospital Asil de Granollers, University of Greenwich, Karolinska Institutet Department of Learning, Informatics, Management and Ethics, University of Piraeus|
|Primary contact||Email: gustavo dot gonzalez at atos dot net|
|Editors||Mylonas, Alexios; Pitropakis, Nikolaos|
|Volume and issue||21(16)|
|Distribution license||Creative Commons Attribution 4.0 International|
|This article contains rendered mathematical formulae. You may require the TeX All the Things plugin for Chrome or the Native MathML add-on and fonts for Firefox if they don't render properly for you.|
Addressing cyber and privacy risks has never been more critical for organizations. While a number of risk assessment methodologies and software tools are available, it is most often the case that one must, at least, integrate them into a holistic approach that combines several appropriate risk sources as input to risk mitigation tools. In addition, cyber risk assessment primarily investigates cyber risks as the consequence of vulnerabilities and threats that threaten assets of the investigated infrastructure. In fact, cyber risk assessment is decoupled from privacy impact assessment, which aims to detect privacy-specific threats and assess the degree of compliance with data protection legislation. Furthermore, a privacy impact assessment (PIA) is conducted in a proactive manner during the design phase of a system, combining processing activities and their inter-dependencies with assets, vulnerabilities, real-time threats and personally identifiable information (PII) that may occur during the dynamic lifecycle of systems.
In this paper, we propose a cyber and privacy risk management toolkit, called AMBIENT (Automated Cyber and Privacy Risk Management Toolkit) that addresses the above challenges by implementing and integrating three distinct software tools. AMBIENT not only assesses cyber and privacy risks in a thorough and automated manner, but it also offers decision-support capabilities to recommend optimal safeguards using the well-known repository of the Center for Internet Security (CIS) Controls. To the best of our knowledge, AMBIENT is the first toolkit in the academic literature that brings together the aforementioned capabilities. To demonstrate its use, we have created a case scenario based on information about cyber attacks we have received from a healthcare organization, as a reference sector that faces critical cyber and privacy threats.
Keywords: toolkit, cybersecurity, privacy, risk assessment, risk control, healthcare
Cyber risk management has traditionally been a fundamental challenge of every organization that seeks ways to protect its assets against cyber threats. This typically involves the use of cybersecurity countermeasures (technical, operational, and physical) to prevent, detect, and respond to cyber attacks prohibiting the exploitation of the organization. Technical controls can be anything from “inventory and control of hardware assets” to “penetration tests and red team exercises,” according to the Center for Internet Security (CIS) Controls. Operational controls refer to standards, policies, and frameworks adopted by the organization, while physical security measures can prevent physical access to the cyber infrastructure.
In most cases, implementing all controls is neither possible nor required, although some controls are necessary for an organization to operate. For example, corporate cybersecurity strategies dictate the need for aligning with information security frameworks such as the International Organization for Standardization's ISO 27001 and 27002 standards. Regarding the different types of organizations, the National Institute of Standards and Technology (NIST) has published the Framework for Improving Critical Infrastructure Cybersecurity, stating that different organizations exhibit different cyber risks due to their different security requirements and infrastructures to be protected. For instance, financial and healthcare organizations have regulatory requirements to satisfy, while the second also have to protect human lives.
Our work is motivated by the need to undertake cyber risk management in the healthcare domain. Nevertheless, cyber risk management methodologies and tools are generally applicable to a variety of industries, with the underlying models and system components largely remaining the same. Our choice was initially motivated by the criticality of this domain determined by cyber–physical impact inflicted by cyber risks as human lives can be at risk following a cyber incident. In the 2020 Data Breach Investigations Report, published by Verizon, healthcare is listed as the industry with the majority of data breaches.
Last year, the same report indicated that healthcare stands out due to the fact that 59% of breaches are associated with internal actors, and 81% of the incidents within healthcare corresponds to miscellaneous errors, privilege misuse, and web applications. Only in the United States, healthcare organizations have reported since 2016 over 170 individual ransomware attacks, affecting around 1,500 healthcare centers and over 6.5 million patients, which represents an estimated cost to the industry of USD $157 million. This rising number of security incidents has also led to data breaches (e.g., 72% of the data breaches were medical), due to the massive amount of sensitive data that is processed. The observations worsen if we look at the currently overwhelmed healthcare domain due to the COVID-19 pandemic.
The well-known WannaCry ransomware, although not targeting healthcare organizations per se, managed to massively affect the U.K.’s National Healthcare System (NHS), posing not only financial damages but also life-threatening ones, e.g., via operations which could not take place when systems went down during the attack. The low security posture of many hospitals was the reason for WannaCry exploiting so successfully its various hosts, causing tremendous impact during a limited period of time. Having a state actor behind this attack is not the only source of danger to healthcare organizations, which may also be susceptible to attacks launched by anyone ranging from script kiddies through to organized crime and state actors.
Even worse, healthcare data are more valuable than many other data in the Dark Web, because of the potential adversarial use of it, including blackmailing to gain some financial gain, selling intelligence to pharmaceutical companies, as well as compromising data integrity to create chaos in a country or a hospital, such as the recent incident at Valley Hospital, in California, which was hit by a ransomware attack on October 11, 2020. The case of WannaCry clearly demonstrated that U.K. hospitals had not invested in cybersecurity controls, while post-incident analysis shows that 65 NHS Trusts spent 612 million pounds on IT two years after the attack took place, which corresponds to 33% budget increase compared to the year preceding this incident.
While everything shows that cybersecurity has been more of an afterthought for healthcare organizations than, for instance, for the banking industry, it is also clear that due to the General Data Protection Regulation (GDPR), hospitals are obliged to report incidents or breaches in data processing. Furthermore, enabling traceability in these domains serves the purpose of demonstrating accountability, which has been recently studied extensively as part of the the literature on blockchain.
Traditionally, information privacy and cybersecurity have been treated as distinct concepts. Even though managing cybersecurity risk contributes to managing privacy risk, it is not sufficient, since privacy risks can also arise by other means unrelated to cybersecurity incidents, while loss of personal data does not equate to a loss of privacy. In data terms, privacy is violated if and only if the data are used in a manner that actually violates the data subject’s fundamental right to privacy. However, as the number of privacy and data protection regulations increase, the overlap between privacy and cybersecurity increases.
Organizations are spending valuable resources by duplicating efforts to mitigate the consequences on privacy and cybersecurity attacks, competing for the same budgets. This brings us to a major challenge of having to spend a proportion of the IT budget of the organization on countermeasures that mitigate cyber and privacy risks. This has given rise to a fairly rich literature of cyber investments seeking answers to what the best ways are to select a portfolio of countermeasures given some predefined financial limitations. Within the same domain, researchers are also investigating the role of indirect costs of these countermeasures to the selection process, how countermeasures interact with each other, and what is the minimal set of countermeasures required to achieved a desirable level of overall risk.
Our work on risk assessment and control has led to the development of an innovative toolkit called AMBIENT (Automated Cyber and Privacy Risk Management Toolkit). Although AMBIENT has been designed based on end user requirements elicited by healthcare professionals, inevitably, its functionalities can be used in other domains. Nevertheless, the knowledge bases of AMBIENT (e.g., vulnerabilities) as well as values for parameters used during risk assessment (e.g., probabilities of attack occurrence) are drawn from the healthcare domain, as published in industrial reports such as the Verizon 2021 Data Breach Investigations Report.
Our motivation behind creating AMBIENT was the lack of automated software that not only conducts cyber risk assessment in the traditional way, but also takes into consideration the GDPR and healthcare processes, and then addresses the fundamental challenge of investing a financial budget to the most effective combination of cybersecurity controls. The automation nature of a cyber risk management tool is critical, because it can save time and resources of an organization that either outsources this task or allocates a significant amount of time to combine the outcomes of the cyber and privacy risk assessments with a tool that suggests best ways to mitigate the identified risks. AMBIENT is a decision support platform that exhibits cyber risk assessment, privacy risk assessment according to GDPR terms and requirements, and proactive cyber risk control (i.e., before threats have materialized) and reactive mitigation (i.e., when signs of intrusions are present or new risks have been identified). At the same time, AMBIENT determines an optimal allocation of a financial budget to various cyber controls by adopting the weakest link model.
AMBIENT is augmented with real-time intrusion detection capabilities to be able to derive changes in the risk that are worthy of being considered by system administrators. Once these notifications are triggered, AMBIENT relies on a Cybersecurity and a Privacy Risk Assessment module, as solutions that take advantage of a variety of input data to perform the analysis and provide qualitative and quantitative scores that will advise organizations on the risks they are exposed to and the mitigation measures they can implement to reduce their attack surface. Such mitigation measures are shared with the Optimal Safeguard Recommendation module that performs further analysis and optimization in order to compute a prioritized list of remediation actions to be taken, acting as a holistic decision support cybersecurity toolkit.
The remainder of this paper is structured as follows. The next section presents the related work in cybersecurity and privacy risk assessments, as well as in optimization of controls. An introduction to the AMBIENT toolkit architecture and details of its main modules follow the related work. Then the applicability of the proposed toolkit is discussed by analyzing security threats in a healthcare infrastructure, followed by a discusses of the preliminary results. The final section of the paper highlights the main advantages and limitations of our proposed toolkit, and provides conclusions as well as perspectives for future work.
Cyber risk assessment
An integral part of the risk assessment process is the selection of a risk assessment model or methodology. There is a vast variety of risk assessment models in the literature and tools available in the market. Examples of models used in quantitative risk assessments include Fault Tree Analysis, Bayesian Networks, Monte Carlo Simulation, and Markov Chains.
Examples of qualitative risk assessment tools include EBIOS RM (Expression of Needs and Identification of Security Objectives), MEHARI (Harmonized Risk Analysis Method), and OCTAVE (Operationally Critical Threat, Asset, and Vulnerability Evaluation). IT-Grundschutz (IT Baseline Protection Manual) is an example of a tool performing quantitative risk assessment. Other tools such as MAGERIT (Risk Analysis and Management Methodology for Information Systems) and CORAS (a method for risk analysis of security-critical systems) are widely used for both qualitative and quantitative risk assessments. Regardless, the choice of risk assessment tool largely depends on the purpose and the data available (e.g., impact, likelihood of occurrence, etc.).
Previous work by Ganin et al. addressed the gap between risk assessment and risk management and how to allow a structured and transparent process of selecting risk management alternatives. The authors proposed a decision-analysis-based approach that quantifies threats, vulnerabilities, and consequences based on multiple criteria to assess the cybersecurity risk levels. The proposed approach provides justifiable methods for selecting risk management actions consistent with stakeholder values and technical data.
Radanliev et al. proposed a model for the definition of individual risks and their measurement. Authors focused on internet of things (IoT) scenarios and integrated an impact assessment methodology to improve understanding of the economic impact values associated with particular devices. As a result, new risk metrics were developed by considering uncertainties and potential challenges specific to the IoT environment. The major limitation of this approach is the lack of evaluation of cyber risks for the unknown but potential vulnerabilities.
Varela-Vaca et al. addressed the problem of automatic security risk management by proposing a risk assessment methodology that enables the analysis and evaluation of multiple activities combined in a business process model to determine the compliance of the model with regards to the security-risk objectives. Authors focused on combining business process management and security-risk descriptions to assess the risk level of the entire process and to identify the risk responsible for a nonconformity. Artificial intelligence techniques were used to automate the presented diagnostic process.
Advances in the area of IoT have brought novel methods that integrate various cyber risk assessment approaches (e.g., Cyber Value at Risk and MicroMort) to compute the economic impact of IoT cyber risks. Recent cyber risk assessment models use a variety of techniques including text mining, fuzzy fractional ordinary differential equations, and Lognormal probabilistic distributions, among others, aiming to rapidly adapt to changing environments and provide accurate risk assessment results.
Privacy risk assessment
The use of a privacy impact assessment (PIA) is considered part of a systematic risk management approach which aims towards an evaluation of potential effects that systems may have on privacy and fostering trust by implementing the privacy-by-design principle. Several standardization bodies and data protection authorities have established legal frameworks and guidelines which mandate the conduction of PIA, among them the GDPR regulation. However, even though the initial notion of a PIA method dates back to 2009, and several published frameworks and guidelines set the principles for the conduction of privacy impact assessment, PIA remains a challenging and difficult process due to the multiple aspects that an assessor needs to consider. According to GDPR, a type of processing is likely to result in a high risk to the rights and freedoms of natural persons, thus the controller shall, prior to the processing, carry out an assessment of the impact of the envisaged processing operations on the protection of personal data. Nevertheless, PIA shall be an on-going process, regularly applied to personal data processing for identifying and mitigating risks in a more dynamic manner.
Privacy data protection standards (e.g., BS 10012:2017, ISO/IEC 29151:2017, and ISO/IEC 27018:2014), can be found in the literature focusing on PIA as a requirement in the execution of cybersecurity risk assessments. PIA and cybersecurity risk assessments are, however, treated as two different and uncorrelated processes, with a clear gap on automated tools, methods, and models that implement PIA. Even though standards (e.g., ISO/IEC 29134:2017) provide details and guidance to conduct privacy impact assessments, they are very generic and provide high-level information that in some cases is insufficient to perform an appropriate privacy risk assessment. Although the literature provides a wide variety of privacy metrics, they mainly consider properties of privacy-enhancing technologies such as the amount of sensitive information leaked or the number of indistinguishable users, instead of the privacy impact. Recently, the NIST proposed a privacy framework in the form of a solid documentation and a practical tool to manage the privacy risks of an organization by prioritizing privacy protection activities through enterprise risk management.  The NIST also developed the Privacy Risk Assessment Methodology (PRAM), which applies the risk model from NISTIR 8062 and helps organizations analyze, assess, and prioritize privacy risks.
In addition, several national regulators have published guidelines for a data protection impact assessment (DPIA), including the French Commission for Informatics and Freedom (CNIL) and the British Information Commissioner’s Office (ICO). Such guidance has been updated to address GDPR’s DPIAs and to provide detailed guidelines about their regulatory requirements and processes. These guidelines follow different approaches and propose diverse steps for conducting PIA. Thus, the adoption of a single methodology becomes a difficult task for an organization, and organizing a PIA project becomes a maze-like process. While there are differences in the aforementioned approaches, they are equally suitable for conducting a DPIA and produce largely similar results.
The ENISA’s on-line tool, which consists of six steps for the calculation of the privacy risk is one of the available PIA tools. The assessment of risks is the first step towards the adoption of appropriate security measures for the protection of personal data. Furthermore, CNIL’s PIA tool considers data controllers that are familiar with the PIA process. This tool lacks automation, in terms of ICT asset inventory and detection of threats or vulnerabilities that can affect privacy, which can increase the awareness of the risk assessor, while the resulted risk levels do not consider the cyber security status of the organization. The GDPR DPIA Tool (DPIA Tool) is a web-based tool for assisting organizations to evaluate data protection risks with respect to GDPR. The tool was developed to support the implementation of DPIA and provides a structured, risk-oriented approach to identification and assessment of potential data protection risks. The structure of the DPIA Tool is based on a questionnaire and thus, it offers a rather limited automation of the assessment of processing activities on personal data within the organization. Last, the Compliance-Kit 2.0 tool follows the British standard BS 10012, GDPR, and ISO 29100, and it is based on the legal obligation to comply with the requirements of GDPR and management’s strategic decisions to implement these regulations with the goal of establishing, maintaining, and developing practical and process-oriented data protection management.
In addition to the aforementioned regulatory efforts, academic research has also proposed improvements to DPIA processes. These efforts include making the DPIA process more systematic and structured by proposing formal modelling techniques for privacy threats. Manna et al., for example, proposed a comprehensive methodology for identifying data privacy risks and quantifying them, while the risk values are computed at different levels to help both senior management and operational personnel in assessing and mitigating privacy risks. Wei et al. proposed a systematic privacy-considered information security risk assessment (pISRA) model, which can take both a privacy impact analysis and risk assessment into consideration. Finally, Henriksen-Bulmer et al. presented an empirically evaluated privacy risk assessment framework based on contextual integrity—the DPIA Data Wheel—that practitioners can use to inform decision making around the privacy risks of cyber physical systems (CPS). However, most of these aforementioned research efforts do not implement their proposed method/model.
Optimal risk control and cyber investments
Controlling cyber and privacy risks is a vital requirement for an organization to become stronger against cyber actors. This is achieved through the implementation of cyber controls, which are always coming with different types of costs, including direct ones (e.g., financial) or indirect (e.g., systems usability). Inevitably, the challenge of improving these risks not only requires ways to optimize cyber control choices, especially by combining these controls to attain greater efficacy, but poses the need for sophisticated cyber investment strategies, first studied by Gordon and Loeb.
The Risk Mitigation component of AMBIENT has been inspired by work published by Fielder et al. and Panda et al. In the latter, we have published the algorithmic side of our component, including the mathematical analysis required to optimize decisions about cyber controls and cyber budget spending. These papers deploy mathematical optimization and game theory to derive optimal strategies for the defending (e.g., the manager of the infrastructure to be protected) and attacking agents (e.g., an adversary). In particular, game theory has been proposed by several works in the cybersecurity literature, e.g., by Rontidis et al. and Panaousis et al.
The seminal works, published by Fielder et al., have proposed novel ways to invest a cybersecurity budget to protect small and medium enterprises against commodity cyber attacks. The authors have used mathematical optimization, game theory, and cybersecurity engineering to assess their framework, without developing a dashboard or an entire software component to offer this advice, as it is the case of the AMBIENT’s Risk Mitigation component.
In a similar vein, Wang studied the tradeoffs between cybersecurity investments in acquiring knowledge and expertise (i.e., personnel) and deploying mitigation techniques. Cybersecurity investments have also been studied as part of supply chain network models. For example, Nagurney et al. empower competing retailers to maximize their expected profits through optimizing product transactions as well as investments in cybersecurity.
Some research has looked into the more specific problem of determining when the best time to invest in cybersecurity is. Chronopoulos et al. proposed a real options approach to tackle this timing problem. By analyzing the cost of cyber attacks and when cyber controls can be deployed by the organization, they derive the optimal timing for such deployment, optimizing returns on security investments.
Beyond investing in cyber controls, some researchers have looked into the effect of uncertainties during the risk assessment phases and how these affect investment decisions. The same works compute optimal strategies given these uncertainties, offering cybersecurity investment models that are robust to these uncertainties, meaning that they optimize return on security investment despite not having the accurate values about the probabilities of different cyber attacks materializing.
Besides studying how to invest in a wide range of cyber controls under uncertainty, Paul and Wang proposed a way to invest optimally between prevention and detection cyber controls. Dutta and Al-Shaer formulate cybersecurity resilience and propose a method to derive the best combination of critical security controls by considering a set of residual cyber risks, the available budget for cyber controls, and resiliency and usability constraints.
In the previous sections we presented a gamut of methodologies, standards, research endeavors and tools related to the three pillars of AMBIENT namely, the Cyber and Privacy risk assessments and optimal safeguards provision. On the one hand, the cybersecurity risk assessment uses a wide base of risk indicators that analyze threat scenarios using a rule matching methodology. This approach enables the competitive advantage of the cybersecurity risk module to operate on top of detection and inventory tools for the conduction of real-time evaluation of cyber risks. In addition, both qualitative and quantitative methods are adopted for risk evaluation. The latter approach is expressed in monetary values and represents the typical loss and the worst-case scenarios to ease decision makers in perceiving the criticality of identified risks. Furthermore, based on its internal modelling, the AMBIENT’s cybersecurity module provides a list of mitigation measures that apply to each risk model to reduce risk levels.
On the other hand, the privacy module aims to bridge the gap between the cyber and privacy risk assessment, which are treated as distinct management processes. The AMBIENT’s privacy risk assessment module operates in tandem with real-time threat inventory tools in order to quantify the privacy impact on the data processing activities of an organization. Thus, our proposed solution exceeds the rigid approaches of privacy impact assessment by utilizing an extendable scoring system and by considering the inter-dependencies of data processing ICT assets (i.e., assets which are engaged in data processing activities) to ease the privacy impact analysis on the fly. As discussed in the next section, the privacy assessment module has the advantage of inter-dependency graphs as the core modelling technique to express the connectivity and relationships between assets, data entries, threats and vulnerabilities in order to identify associated risks. Thus, the inter-dependencies assist not only on the the data flows representation, but is also used to define the processing activities and the possible attack paths for privacy risk calculation. Yet, we offer a high level of automation, which has been documented as a gap in the state of the art of PIA conduction. Overall, in order to meet the requirements of the regulatory frameworks and be in line with the requirement to increase automation in supporting the PIA processes, the privacy risk assessment module of AMBIENT offers the following advantageous characteristics:
- asset inter-dependencies documentation;
- data processing flows identification;
- automation and dynamicity of PIA;
- cyber risk consideration in PIA;
- mitigation controls;
- GDPR PIA support; and
- automated privacy impact scoring.
AMBIENT’s risk mitigation module offers strategic decisions on cybersecurity countermeasures and investments based on the finding of the cybersecurity and privacy modules. The game theoretic approach of the risk mitigation component and its optimization methodology offers decision support considering both the efficacy of the controls and the cost of implementation to infer on their optimality for mitigating the identified risks. In addition, the proposed solution takes advantage of the use of the well-documented basis of CIS controls and offers a database of tools that are recommended to the user as part of an optimal cyber strategy.
Furthermore, apart from the advancements brought by each tool individually, AMBIENT unifies the described functionalities under the same umbrella by providing a solution that can operate with a high level of automation and can support decision makers to the maze-like risk management and mitigation operations. To the best of our knowledge, AMBIENT is the only solution that offers this pipeline of tools that brings together the cyber and privacy risk assessment for real-time evaluation, and bridges the gap of responding to the identified cyber and privacy threats based on strategic investments. Through this synergy, AMBIENT can satisfy the real needs of vertical sectors that present a constantly changing cyber and privacy threat surface, such as the healthcare sector.
AMBIENT: Automated Cyber and Privacy Risk Management Toolkit
As depicted in Figure 1, AMBIENT is a toolkit composed of three main modules:
- the Cybersecurity Risk Assessment module, aiming to assign qualitative and quantitative risk levels to potential cyber threat scenarios;
- the Privacy Risk Assessment module, which is responsible for analyzing potential privacy threats; and
- the Risk Mitigation module, which is responsible for evaluating, ranking, and selecting optimal security measures to mitigate cyber risks.
This section goes on to describe all modules of the AMBIENT toolkit. The toolkit receives infrastructure data (e.g., monitoring data, vulnerabilities, system configuration, risk models) and produces qualitative and quantitative risk scores that indicate the cybersecurity and privacy risk levels of an organization, a business unit, or an individual asset. Risk scores are accompanied by a list of mitigation measures that are ranked according to multiple factors, and that indicates the priority to be given to their implementation.
Cybersecurity Risk Assessment module
The Cybersecurity Risk Assessment module is in charge of analyzing data aggregated from multiple sources and assessing the risk level of an organization. This module uses incident detection functionalities, offering the capabilities of a security information and event management (SIEM) solution , which can handle large volumes of security data. It produces both qualitative and quantitative cybersecurity scores of the risks which an organization are exposed to. The cybersecurity risk assessment focuses on collecting and analyzing cybersecurity events (e.g., malicious incidents) in real-time (or near-real time) and correlating them to be used as an input to the risk assessment. In addition, this module provides useful information (e.g., risk levels, potential financial losses, worst case scenarios, mitigation measures to be implemented) to help C-level managers thus improve cybersecurity awareness on topics related to information technology (IT) and operational technology (OT). Furthermore, this module is able to capture information from cybersecurity events (e.g., detected threats, attacks, potential incidents) through the use of cyber agents deployed on the organization's infrastructure. The information processed by this module can be accessed by a visualization framework that presents the main outcome in a dashboard for monitoring and response, which further helps on the generation of cybersecurity alarms and risk reports.
The CORAS tool is used to generate graphical risk models for the risk analysis process. An important part of the risk modelling stage is to create human-readable files, which provide the graphical representation of the risk models and serve as intermediate points to create the corresponding algorithms executing such models. Each CORAS risk model uses both measurable (R and non-measurable (DEXi) assessment algorithms depending on the information each algorithm provides. The programming language R is used to produce monetary risk reports for each target and risk. This algorithm is also a conversion of the risk models to R scripts to supply economic loss estimations. DEXi is used to produce qualitative overall reports using fuzzy logic and generate assessments for each target and risk within a risk model, as well as the platform as a whole. The algorithm represents a qualitative adaptation of risk models to DEXi scripts, which will be executed by the cybersecurity risk assessment module. Using both approaches helps AMBIENT to improve its analysis by adding complementary information about the risk level and potential consequences of a given threat if realized on the targeted system.
Besides qualitative and quantitative scores, a list of mitigation measures is provided for each risk model, and if these are implemented, risk levels should be reduced to acceptable values. This list of measures is produced as a result of lookups on tables that correlate threat indicators to cyber controls. Nonetheless, AMBIENT does not enforce the mitigation measures; it is just a decision support toolkit that helps security administrators and security managers, like chief information security officers (CISOs), to define mitigation strategies based on the computed scores.
Qualitative risk assessment is computed as the probability of a threat exploiting a vulnerability (i.e., Likelihood) times the consequences of such vulnerability being exploited (i.e., Impact), as typically expressed in the literature by the formula . The latter is computed using a set of risk indicators that analyze a particular threat scenario based on a rule matching methodology. (A concrete example of a quantitative risk assessment is detailed in the next section.) This module evaluates the risk level of the monitored infrastructure through the use of different algorithms whose inputs are provided in the form of indicators. These latter are built based on the values of different sources of information (e.g., vulnerability assessment tools, SIEM environments, end user’s business profile). This module uses two main data formats: Indicators, to refer to pre-conditions taken from risk models when compared with monitored input data; and Indicator Values, to refer to the real input required to compute the cybersecurity risk scores.
The Cybersecurity Risk Assessment module has the following components:
- Indicator Value Generator: used to collect all inputs from the external sources of information (e.g., questionnaire data, target information, vulnerabilities)
- Triggering Detector: receives new or updated indicator values and the target information related to the loss estimations, as well as the risk models selected for the assessments; invokes the Risk Model Executors upon a change in any of its inputs
- Instantiator: in charge of creating an instance (i.e., qualitative used by DEXi, and quantitative used by R) of the risk models using the current indicators values received as inputs
- Model Rules Executor: performs two simultaneous analysis: (i) the qualitative risk assessment for the corresponding target and the risk model using the DEXi Model Rules Executor, and (ii) the quantitative risk assessment for the corresponding target and risk model using the Model Rules Executor
- Aggregator: aims to group the individual risk assessment of an organization per asset (e.g., workstation, server, printer, cellphone) per risk model (e.g., Denial of Service, Bypass Login, Cross-Site Request Forgery) and/or per security attribute (i.e., confidentiality, integrity, availability)
- Data Warehouse: represents the central data storage component, which stores the following information: (i) users and organizations (input manually by administrators); (ii) users’ configuration parameters (input by end users); (iii) risk models; (iv) catalogues of risks, mitigation measures, and indicators; (v) risk reports (results of finished risk assessment procedures); (vi) active deployed sensors; (vii) events reported by sensors; (viii) alarms reported by the Monitoring Engine; and (ix) vulnerabilities found by the vulnerability scanners
Input and output data processed/generated by this module are summarized in Table 1 and Table 2, respectively.
Privacy Risk Assessment module
The Privacy Risk Assessment module is the primary tool for evaluating the privacy risk level of an organization. This module considers the current cybersecurity status acquired by the cybersecurity Risk Assessment module and performs analysis of data processing operations to uncover potential privacy risks—in alignment with the GDPR objectives—and protect sensitive user data. More specifically, it performs privacy risk analysis based on cybersecurity evidence coming from the deployed infrastructure sensors and the documented data processing actions of the organization. To perform this, the privacy risk assessment considers the interrelations that exist among information and communication technology (ICT) infrastructural assets that support data processing activities, data sources, data subjects and personally identifiable information (PII) and infers the privacy risks that an organization may face due to vulnerabilities and threats targeting it. The privacy assessment process results in a list of potential mitigation measures which could be enforced by the security administrators. These mitigation measures can only be used to help cybersecurity decision-makers (e.g., CISOs) to define mitigation strategies based on the computed privacy scores. The functionality of the privacy risk assessment is based on two pillars:
- a novel and extensible privacy risk scoring system for quantifying the privacy risks imposed by quantitatively scaled identified vulnerabilities and threats that have an impact on privacy when targeting assets; and
- a dynamic and extensible system model that maps core GDPR entities and requirements for assisting the information security decision makers in keeping track of all risk-related information and assessing the degree of compliance of the organization.
Together, these two pillars create a competitive advantage, which is the ability to consider the cybersecurity status of an organization and quantify the privacy risks in complete alignment with GDPR requirements.
The Privacy Risk Assessment module has the following components:
- Data Warehouse and Model Initialization: These two components are closely related to each other. Data Warehouse is a NoSQL database based on MongoDB that stores all the external input, while the Model Initialization component “translates” the stored information, consults the different modelling methods used in the Privacy Risk Assessment module, and directly feeds the corresponding components.
- Asset Modelling: This component uses the inter-dependency graph approach introduced by Polemi and Kotzanikolaou The nodes represent the individual assets and the edges represent the inter-dependencies amongst them. Such a graphical representation model is a cornerstone in the Privacy Risk Assessment module, as it works as the “glue” that keeps together ICT assets, data entries, threats, and vulnerabilities in order to identify the risk data processing activities of an organization. This component uses the inter-dependency types IsConnectedTo, IsUsedBy, IsProcessedBy, IsLocatedIn, IsStoredOn, and IsInstalledOn to annotate the relation among assets. These relations are not used only to denote connections among tangible ICT assets, but also to intangible ones, such as data, health records, and PII. Overall, by utilizing the inter-dependency graphs, a security analyst is in position to identify potential privacy risks based on a cartography of assets, which encapsulate their vulnerabilities and the potential privacy threats posed against them. In this way, the inter-dependency graphs contribute not only to the uncovering of privacy risky individual assets, but crucially, they ease in highlighting risk-prone privacy paths which are formed by chains of assets included in a specific processing activity.
- GDPR Modelling: The management of processing activity is the principal aspect of GDPR Modelling, aggregating all the GDPR-related information. The main information that processing activity includes can be divided in three parts: the processing purpose and the involved entities, all the processed personal data assigned to specific subjects, and the asset chain that is involved in the processing activity. By combining the aforementioned elements, the security analyst is in a position to consolidate all the necessary information for processing activities, including the engaged supporting ICT assets, and define the dependency with intangible personal and sensitive data assets.
- Considering that information systems may store and process a huge amount of data, GDPR Modeling adopts a specific data categorization, as the criticality of the data is not always the same. In fact, the categorization of personal data is considered essential, as some processing activities may focus on publicly available data, while others focus on financial or even sensitive data. This indicates the need to assign a different criticality level to the aforementioned data types and treat personal and sensitive data as data types that can clearly have a higher impact on the fundamental rights and freedoms of the individuals in case of data breaches. That is, AMBIENT identifies the following categories based on the classification proposed by Makri et al. and assigns different criticality scores according to the scoring methodology presented by Papamartzivanos et al.:
- sensitive personal data (e.g., medical data, legal documents);
- personal data (e.g., data which uniquely identify a person, such as IDs, Social Security Number [SSN], personal or marital status);
- financial data (e.g., data related to financial transactions, accounting entries);
- operational data (e.g., data generated during the execution of a service, log files); and
- other data (e.g., data that cannot be classified in any of the above categories, and belong to a lower criticality level).
- In practice, it is up to the data protection officer (DPO) or the security administrator to identify the correct data class when instantiating AMBIENT in the context of the identified processing activities of the organization.
- Privacy Threat Modeling: This component, as its name suggests, aims to provide the threat characterization score. Given the information of quantitatively scaled identified vulnerabilities and threats, this component facilitates the privacy scoring calculation based on the type of the threat, the sensitivity of the corresponding vulnerable asset, and the calculated cybersecurity risk score. The aforementioned factors contribute to a formula inspired by QED Secure Solutions and Whitescope, in order to reflect the impact that a cyber threat may have to the data protection and privacy dimension.
- Privacy Impact Assessment: The Privacy Impact Assessment component aggregates all the information from the modelling components and undertakes the calculation of the privacy scores. These scores are calculated on an asset basis and quantify the impact that a vulnerability or a threat may have due to the affected asset, which is used to support data processing activities. Given the severity of the threat and the peculiarities derived from the Privacy Threat Modeling component, the Privacy Impact Assessment component assesses the impact on fundamental rights and freedoms of the individuals, following the classification used by The European Union Agency for Cybersecurity. The privacy scoring system combines the threat characterization and the privacy impact. The scoring system uses a weighted scale to focus on the impact to users’ privacy while considering the threat. However, the exact value of the weights is a parameter that can be adjusted accordingly, given the preferences and the domain knowledge of experts in different sectors. The weighted scale formula is given by the formula . More details on the idea behind this weighted formula can be found reviewing Papamartzivanos et al.
- Privacy Risk Quantification Engine: The Privacy Risk Quantification Engine is as sub-component of the Privacy Impact Assessment component, providing three different privacy scores: the asset-level privacy score, the processing activity-level privacy score, and the organization-level (global) privacy score.
It must be noted that both the Cybersecurity Risk Assessment and Privacy Risk Assessment modules consider the same cybersecurity vulnerabilities identified by the same external tool (e.g., OpenVAS). However, the difference is that the Privacy Risk Assessment module categorizes and prioritizes the vulnerabilities by using a privacy-oriented approach based on the type of vulnerability. For instance, the privacy score is higher when the confidentiality of data may by affected (e.g., SQL injection attack) and lower when the availability of a system is affected (e.g., DoS attack). Input and output data processed and generated by this module are summarized in Table 3 and Table 4, respectively.
Risk Mitigation module
Mitigating cyber and privacy risks is one of the major outcomes of information security management. This mitigation may be preemptive or reactive. By preemptively choosing cybersecurity controls, organizations reduce the likelihood of attacks exploiting their assets and causing devastating impact. The controls act as methods to improve the current security level of an organization and are usually instances of well-known security frameworks such as NIST SP 800-53 Rev. 5 Security and Privacy Controls for Information Systems and Organizations, CIS Controls, and ISO 27001 controls.
The Risk Mitigation module of AMBIENT supports organizations with mitigating risks by addressing both cyber strategic decisions (called "CHANGE" in CISO language) and more immediate security actions (called "RUN") system administrators or the security team will benefit from implementing. It computes optimal defensive plans of cybersecurity safeguards for decision-support, which advises cybersecurity decision-makers on how to combine various safeguards to minimize overall cyber–physical risks threatening an organization. It comprises the Core and the Dashboard; the former implements models of cybersecurity control optimization and cyber investment, while the latter visualizes the performance of the selected safeguards and the decision support guidance. The Core generates all required data to be visualized by the Dashboard. The model and mathematical frameworks used in the Core are replicated from Panda et al.
The overall aim of the Risk Mitigation module is to act as a decision-support tool for cybersecurity decision makers and help them:
- determine long-term best cybersecurity strategies—in the form of advice—in terms of mitigating cyber and privacy risks subject to financial constraints by using fundamental principles of cybersecurity risk management to create the Core model and multi-criteria mathematical optimization to solve the underlying decision-making challenge;
- visualize the cybersecurity advice using the CIS Controls v7.1 (a well-known framework of cybersecurity safeguards) by generating practical and detailed advice on tools and processes required to implement the safeguards and visualizing the performance of cyber controls in terms of risk mitigation;
- visualize the results of risk improvement to raise awareness of decision makers on how each cybersecurity safeguard improves the security posture of the organization by using the Dashboard; and
- prioritize short-term cyber actions that the organization must take against specific cyber threats and risks identified by the cybersecurity and privacy risk modules.
The above objectives are also aligned with best practices for cybersecurity in procurement, as published by the European Union Agency for Cybersecurity. The cybersecurity strategies consist of combinations of cybersecurity controls drawn from a well-known repository. The Risk Mitigation module uses the framework of CIS Controls, which comprises a well-known repository of cyber controls based on two items: real attack data, and a consensus development process involving cybersecurity experts to create a prioritized list of actions that increase the cybersecurity level of an organization mitigating both vulnerability-based and threat-based risks.
The Core consists of six components that all work together to deliver a set of risk control recommendations. These offer actionable advice on what cyber controls to implement and how they perform in terms of costs and risk reduction. The advice is visualized to the user through the Dashboard. The core parts of the Risk Mitigation module are:
- Risk Parameters Initializer: This component initializes the parameters required to compute the optimal set of safeguards that mitigate cyber and privacy risks, including the reports received by the risk assessment modules.
- Safeguard Game Generator: This component uses AI optimization generating a strategic game between a defending and an attacking agent based on the game-theoretic concepts used to compute equilibria, i.e., optimal points. This game is represented by the available actions of the agents and their payoff functions. The defending agent can choose among different ways of implementing a cyber control, and the attacking agent can choose among different attack methods.
- Safeguard Game Solver: This component calculates the game equilibria, which are optimal combinations of implementation ways (can be seen as levels when the ways refer to different intensity of implementing the control) chosen for each control used to mitigate cyber and privacy risks against the attacking agent. The Risk Mitigation module uses the 20 CIS Controls, which include 171 sub-controls. This component solves the game for each of these sub-controls to create a repository of optimal available safeguards.
- Cybersecurity Budget Distributor: This component takes a budget and distributes it among all the 20 CIS Controls, which is then allocated to its safeguards.
- Combinatorial Safeguards Generator: For each CIS control, this component generates all the combinations of safeguards, which have been calculated previously by the Safeguard Game Solver.
- Safeguards Plan Solver: For the budget allocation derived previously, this component calculates the safeguard combination that fits into the available budget of the defending agent and minimizes the maximum risk inflicted by the attacking agent respecting the “weakest link” concept.
The Dashboard is used for three main high-level purposes: to communicate to the user the optimal risk control recommendation; to visualize its performance, in terms of risk reduction; and to visualize the different costs, direct and indirect, of the safeguards included in the recommendation.
Input and output data processed and generated by this module are summarised in Table 5 and Table 6, respectively.
This section presents one threat scenario composed of two attacks: an SQL injection and a ransomware attack, both mitigated by AMBIENT. The scenario evaluates concerns with the cybersecurity and privacy awareness level of the organization regarding data exchange in remote healthcare services, as inspired by the use cases of the H2020 CUREX project.  Cybersecurity and privacy risk assessments are conducted by the Cyber Risk Assessment and Privacy Risk Assessment components of AMBIENT, and then optimal security measures are proposed by the Risk Mitigation component.
Point-of-care (PoC) systems have been widely used in hospitals in order to provide innovative solutions to medical professionals and physicians and provide them with an overview of the patients’ condition in a way that makes it easier for them to respond on time and prevent critical situations. PoC systems are platforms that incorporate devices and applications in order to collect, process and visualize data. Naturally, these types of platforms represent an expanded attack surface, as the variety of devices and systems used have unique vulnerabilities which can be challenging to identify and address. With an ever-increasing amount of data containing personally identifiable information and sensitive medical data being communicated across various devices, back-end analytic platforms, user workstations, smartphones, or sensors, it becomes evident that there are multiple threats that can breach legitimate systems and data. Hospitals and care centers need to address such cyber–physical challenges by efficiently assessing the associated risks and mitigate them with appropriate cybersecurity safeguards.
As depicted in Figure 2, and for the purpose of demonstrating the competitive advantages of AMBIENT, we have considered a real testbed composed of a subset of the assets and devices used in the Spanish Hospital Fundació Privada Hospital Asil de Granollers (FPHAG), which includes the following elements:
- SAVAC client application: Either accessed from the CITRIX server farm or from a PC that has the client version installed locally and connects to the SAVAC database, installed in the hospital’s data center (DC).
- Workstations: Three PCs are placed inside the users’ VLAN with the basic programs and hospital’s user credential handling procedures.
- SAVAC Database server: Consists of a cluster of servers that contains all the information stored.
- Firewall: The hospital’s DC is generally supervised by a firewall system in which specific rules are programmed. Virtual LAN users, servers, and devices are bi-directionally connected to the hospital’s firewall.
- Switch: This network component has a dedicated link to the cluster of servers and another to the rest of the network’s elements.
- Analyzers: The hospital uses analytic platforms, which operate directly on data collected by SAVAC and generate analytic dashboards and visualization reports for the hospital managers.
- PACS image server: The images are stored on picture archiving and communication system (PACS). To retrieve the images, a call is made from SAVAC to a URL through a unique identifier of the patient’s image study.
- Integration server: This is used to collect all data and external files and integrate them into SAVAC, either in the database or on the file server, or by external links using identifiers, as in the example of the PACS image server.
- Medical equipment: Different networked medical devices are placed in VLAN.
- Smartphones: Smart devices connect through the hospital’s Wi-Fi (open Wi-Fi validated via capture portal), using a specific identification that the hospital’s firewall allows to be visible, to operate and to have internet access.
In the testbed infrastructure depicted in Figure 2, the PCs run under the Windows 10 operating system; the servers are then placed in a VMware, while the medical devices will run in different versions of Windows and Linux operating systems. In addition, the mobile application to be assessed will run on an Android v9 smartphone. More information about the testbed and use case scenarios can be found in the work of Jofre et al.
Use case scenarios
AMBIENT has been designed to be used in the healthcare domain for a variety of use case scenarios related to health data exchange. Examples of use case scenarios where the toolkit can be useful include the following:
- cross-border patient data exchange, originated when a patient from hospital A has a malaise in a foreign hospital (i.e., hospital B), and due to the emergency, hospital B requests a transfer of a copy of the patient’s health records to hospital A;
- data exchange in mobile healthcare platforms, which considers malfunctioning IoT devices that fail to register measurements, which imposes not only service disruption but also threatens to compromise the patients’ safety and health;
- data exchange in remote healthcare services, which includes threats related to the confidentiality and integrity of the patients’ data coming from healthcare devices and applications (e.g., mobile applications for collecting blood pressure, health rate, temperature, etc.); and
- data exchange for healthcare research, which includes privacy challenges originated from the exchange of health data for research purposes with third parties such as universities and research groups.
In addition to health data exchange scenarios, AMBIENT is able to analyze a wide variety of threats affecting the appropriate operations of healthcare organizations. Such threat scenarios are derived from risk patterns that include inputs related to different vulnerabilities, incidents, and/or infrastructure context that may cause an undesirable situation with a certain likelihood, and which may have consequences in the risk level and economic loss in terms of confidentiality, integrity, and availability. Examples of these risk patterns include a denial of service attack, invalidated redirects and forwards, bypassed logins, compromised security via Trojan-malware, client-server protocol manipulation, session fixation, cross-site request forgery, and SQL injections.
CVE and threat model selection
A vulnerability analysis performed in the target infrastructure identified a list of common vulnerabilities and exposures (CVEs), from which the hospital security team decided to analyze those with critical severity (i.e., using a Common Vulnerability Scoring System [CVSS] higher or equal to 9.0). As a result, two critical vulnerabilities have been identified in some of the hospital’s assets, the exploitation of which could directly affect its IoT medical devices (i.e., MEDEV01 and MEDEV02):
- CVE-2020-11896, which allows remote code execution related to IPv4 tunneling; and
- CVE-2019-11510, which allows attackers to remotely access the targeted network and perform arbitrary file reading.
The hospital’s IT department has raised concerns about the cybersecurity and privacy issues that may emerge from the operation and the communication of the clinical data. Since the data contain highly sensitive PII, it must be ensured that the hospitals’ information systems are properly maintained, and any vulnerabilities are identified and patched in a timely fashion. In addition, since the hospital has the technical capability of generating data reports and exchanging them with third parties, the platform must ensure that proper cybersecurity and privacy safeguards are in place in order to protect the integrity of the data and, most importantly, the patients’ safety. Consequently, the hospital integrates AMBIENT to perform a cyber and privacy risk analysis in order to immediately address risks that exceed the acceptable levels.
Two threat scenarios have been associated to the critical vulnerabilities found in the hospital: an SQL injection (exploiting CVE-2020-11896 via a remote code execution), and a ransomware attack (exploiting CVE-2019-11510). The main concern about an SQL injection attack is that it could allow the intruder to change, delete, and add patients and hospital information and cause malfunctions on the regular procedures. A security compromise via ransomware is a user level threat, which can potentially give access to an intruder to the forms and tests of patients stored in the hospital servers. A negative concern about this threat is the possibility to allow the intruder to encrypt a vast part of the patients and hospital information processed by the medical devices (e.g., MEDEV01, MEDEV02) and ask for a payment to rescue the information and/or consequently stop most of the hospital ongoing activities. These types of attacks can have a greater impact on users’ privacy, since the attacker can have direct access to the sensitive data, in contrast to denial of service (DoS) attacks that mainly affect the availability of the asset.
Cybersecurity and privacy risk assessment results
The previous report indicates that the risk model WPR4, that corresponds to a security compromise via Trojan-malware (i.e., ransomware), and the risk model WPR8, that corresponds to an SQL injection, are assessed as high and very high, respectively, with potential damages that could range from 127,826 EUR to 1,756,863 EUR (as depicted in Figure 3). For these threats, we have identified a set of mitigation measures and we have assessed their efficacy and cost in this scenario, based on the expert knowledge of the end user’s team.
Similarly, the privacy risk module generates the output of the privacy assessment process, as depicted in Table 8. As already mentioned, three different privacy risks scores are generated, namely the privacy score per asset, per processing activity, and the global privacy score of the organization. The global privacy score is based on the maximum of scores of the processing activities, while the score of a processing activity is the maximum of the included assets. The representation of the aforementioned outputs would result in a lengthy technical documentation that goes beyond the scope of this paper.
Table 8 indicates the privacy risk analysis performed by the Privacy Risk Assessment module over the potential SQL injection and ransomware attacks. The risk level assigned to the organization is a product of the analysis performed on the defined data processing activities of the organization given the set of the supporting assets affected by the SQL injection and ransomware attack. Figure 4 displays the main dashboard with the calculated global privacy score and some additional statistics. From a qualitative perspective, it assigns a "Very high" level of risk, while 9.8 out of 10 for the quantitative one. The privacy risk is calculated considering the peculiarities of each case, i.e., the sensitivity of the asset, the vulnerability type, the type of processed data, and the number of the affected processing activities. More details regarding the quantification methodology and formula can also be found in the work of Papamartzivanos et al. Given the great magnitude of an SQL injection against the PREPACSQL asset, which is used to store and process sensitive personal information of patients, the privacy risk model considers the interdependencies and derive the aforementioned privacy risk level.
The joint risk computed from the cybersecurity and privacy results in a "Very high" level, which requires the implementation of security measures before undertaking any data exchange with third-party organizations.
Risk mitigation results
This subsection provides details about the reactive and preemptive controls used by the risk mitigation module of AMBIENT to select optimal mitigation strategies against the analyzed threat scenario.
For this threat scenario, we have identified a set of mitigation measures and have assessed their efficacy and cost based on the expert knowledge of the end user’s team. Table 9 shows the suggested countermeasures to address the identified risks. We have used the Risk Mitigation component of AMBIENT to prioritize the list of remediation actions given that each of them has a benefit and a cost. This component can also be used to provide a cyber strategy of the organization given the aggregated risks identified by the cybersecurity and privacy risk assessment modules and offering preventative cyber and privacy risk reduction capabilities.
Based on the information associated to each mitigation measure, the Risk Mitigation module generates the JSON file—depicted in Table 10—as an output of its analysis.
Preemptive controls subject to a budget
Given a specific cybersecurity investment budget, AMBIENT can derive the best combination of cyber controls by using the Risk Mitigation module. This is run for a budget of EUR 20,000 to be spent on implementing CIS Controls. Figure 5 provides examples of the risk mitigation output data for the CIS Control 8 “Malware Defenses.” We notice that the tool has selected the sub-control “Centralize Anti-Malware Logging,” which implements the capability of “sending all malware detection events to enterprise anti-malware administration tools and event log servers for analysis and alerting.”
The advice suggests that by implementing this sub-control, the organization will benefit from improvement in various different areas of threat prevention such as DoS, malware, and man-in-the-middle attacks. The direct cost is also presented there, split into the different network layers (Demilitarized Zone-DMZ, Intranet, Private Subnet) for this specific sub-control in this organization.
Figure 6 provides examples of the Risk Mitigation dashboard, which outputs data presenting the cost levels for the proposed CIS 8 Control “Malware Defenses” using 8.6 “Centralize Anti-Malware Logging.”
Discussion and conclusions
Most risk assessment tools found in the literature either perform a cybersecurity or a privacy risk assessment. Most tools on the market have a rather narrow scope of application, with a single use case being the norm. There exists a considerable number of tools supporting the documentation of data processing practices, the formulation of consent templates, or the documentation of privacy and data protection policies. Apart from CNIL’s PIA tool, available methods make no reference to any tools that can automate the PIA process or create a PIA report. While, the cybersecurity status of the organization in which the impact analysis is performed is largely neglected by those tools.
Furthermore, the current state-of-practice on risk assessment and analysis of data-related vulnerabilities in the healthcare domain includes mostly custom and proprietary solutions (tools, mechanisms, techniques, procedures) that are typically employed on demand, e.g., when new systems or components are installed, or when new policies need to be enforced. Most standards specify framework conditions for the risk management process but rarely go into detail of specific methods for the risk analysis or risk assessment. This is one of the reasons why often differences in the risk assessment arise within specific areas of application, such as in healthcare, making a direct comparison of the results cumbersome.
In addition, the interoperability and information sharing among the underlying security-related components (usually coming from different technology providers and vendors) is hard to achieve, which in turn increases operational costs and stress during the daily tasks in hospitals and care centers. As a consequence, the healthcare sector is still far from a unified framework that will address vulnerability and risk assessments and analysis through a holistic solution that will compose and orchestrate the heterogeneous tools, mechanisms, techniques, and procedures while at the same time respecting privacy and adhering to GDPR policies.
Regarding limitations of the toolkit, in order to perform the cybersecurity risk assessment, AMBIENT requires a minimum set of input data from the healthcare organization. If no vulnerability is detected or no business profile is provided by the healthcare organization, our proposed toolkit will not be able to compute the cybersecurity risk score. The qualitative and quantitative risk scores associated with the SQL injection attack discussed in the previous section evaluate a threat scenario that considers eight indicators (see Table 11), as well as the identification of potential vulnerabilities and weaknesses. The exploitation of these weaknesses will potentially lead to four unwanted incidents (i.e., privilege escalation by an attacker, unauthorized reading of data, unauthorized code execution, and unauthorized modification of data). Such incidents, if realized, will negatively impact confidentiality, integrity, and availability of the infrastructure’s assets, and hence privacy.
Limited input data will lead to inaccurate cybersecurity and/or privacy risk scores which will affect the selection of optimal mitigation measures. Cybersecurity and privacy scores are therefore highly dependent on the input data provided by the target infrastructure. Similarly, the risk mitigation module highly depends on the input data provided by the cybersecurity and privacy modules about the potential mitigation measures for each threat scenario.
In addition, it is important to mention that AMBIENT is a decision support tool that provides advice and guidelines to security analysts and administrators on the basis of the potential risks affecting their infrastructures, but it does not implement automatic security actions to reduce the organization’s risk level. Security administrators and C-level managers should decide which strategy to deploy, and use the AMBIENT outcome as a guide in their decision-making process. The implementation of a mitigation measure therefore requires manual intervention from the healthcare infrastructure and will generate an effect on the cyber climate, as it will change indicator values in the system, and consequently, cybersecurity and privacy risk scores are expected to decrease. Mitigation measures are displayed in a simple way in the toolkit dashboard (including an ID, the name, and a description of the mitigation measure), and can also be shared in the JSON format.
Another important aspect in the risk assessment process is the time spent for AMBIENT to compute their results. In general, scores are generated within a few minutes (depending on the type of threat analyzed and the number of indicators to compute), and mitigation measures are analyzed within seconds (right after the safeguard candidates are generated by the cybersecurity and privacy modules). The cost/benefit information associated with each mitigation measure is input data obtained through expert knowledge from the healthcare technical personnel. The accuracy of the results provided by risk mitigation module depends on the reliability of the provided input data, and the statistical and mathematical model used in the evaluation. It is worth noting that the risk mitigation part of AMBIENT is able to analyze not only reactive mitigation measures but also proactive mitigation measures. The former are immediate actions required to eliminate or reduce the risk level of a given organization, whereas the latter refers to medium or long-term actions required to reduce the attack surface of the target organization. The two threat scenarios presented in the previous section only include reactive mitigation measures, as we have assumed that the risk assessment is requested after a security event has been detected on the system. AMBIENT relies on this technology to enforce GDPR and record the risk assessment reports. As a result, the proposed approach will control the health data exchange process by reinforcing security, improving traceability, and improving auditability functionalities.
This paper described AMBIENT (Automated Cyber and Privacy Risk Management Toolkit) that evaluates and analyzes the cyber and privacy risks of an organization and recommends mitigation measures that maximize risk reduction given a knowledge base of countermeasures, along with their direct and indirect costs, and subject to a financial budget. The proposed toolkit is composed of three main modules: a Cybersecurity Risk Assessment module, responsible for analyzing potential cyber threat scenarios; a Privacy Risk Assessment module, responsible for analyzing potential privacy risks in alignment with the GDPR objectives; and a Risk Mitigation module, responsible for evaluating and selecting optimal measures to mitigate selected risks.
AMBIENT was deployed in a healthcare infrastructure to evaluate different attack scenarios potentially affecting their daily operations. As such, AMBIENT has been created to support cybersecurity decision makers with cybersecurity and privacy assessment identifying critical assets, potential threats to face, consequences that these threats may cause if they occur, and the actions to be implemented for their mitigation.
As a decision support tool, AMBIENT provides advice on the basis of the potential security and privacy risks affecting target infrastructures, however, it does not automatically implement mitigation actions. Security managers can use AMBIENT’s results as a guide in their decision-making process to define appropriate security policies and strategies that keep risk scores within acceptable levels. Future work will focus on the integration of additional threat and vulnerability frameworks towards a more holistic risk management suite of tools. Additionally, exploring novel privacy-preserving techniques in the context of blockchain applications is left as future work which could bring more possibilities to extend the use of AMBIENT. It will also investigate the applicability of the toolkit to mitigate social attacks, which represent the highest risk to organizations and appear in the most claims as reported by cyber insurers.
G.G.-G. performed conceptualization, methodology, software, validation, formal analysis, investigation, writing original draft, writing—review & editing, visualization, supervision. S.A.M. performed conceptualization, methodology, software, validation, formal analysis, investigation, writing original draft, writing—review & editing, visualization, supervision. D.P. performed conceptualization, methodology, software, validation, formal analysis, investigation, writing original draft, writing—review & editing, visualization, supervision. R.R. performed validation, resources, writing original Draft. D.N.-L. performed validation, resources, writing original Draft. C.O. performed conceptualization and methodology, paper editing. S.N. performed conceptualization and methodology and paper editing. C.X. performed conceptualization, methodology, investigation, writing—review & editing. E.P. performed conceptualization, methodology, validation, formal analysis, investigation, writing original draft, writing—review & editing, visualization, supervision, funding acquisition. All authors have read and agreed to the published version of the manuscript.
The research work presented in this article has been supported by the European Commission under the H2020 Programme, through funding of the “CUREX: seCUre and pRivate hEalth data eXchange” project (with Grant Agreement number 826404).
Conflict of interest
The authors declare no conflict of interest.
- Whitman, Michael E.; Mattord, Herbert J. (2012). Principles of information security (4th ed ed.). Boston, MA: Course Technology. ISBN 978-1-111-13821-9.
- Center for Internet Security. "CIS Controls". https://www.cisecurity.org/controls/. Retrieved 31 May 2021.
- International Organization for Standardization. "ISO/IEC 27001 Information Security Management". International Organization for Standardization. https://www.iso.org/isoiec-27001-information-security.html. Retrieved 14 June 2021.
- Kruse, Clemens Scott; Frederick, Benjamin; Jacobson, Taylor; Monticone, D. Kyle (1 January 2017). "Cybersecurity in healthcare: A systematic review of modern threats and trends" (in en). Technology and Health Care 25 (1): 1–10. doi:10.3233/THC-161263. ISSN 0928-7329. https://content.iospress.com/articles/technology-and-health-care/thc1263.
- Verizon (2020). "2020 Data Breach Investigations Report" (PDF). Verizon. https://enterprise.verizon.com/resources/reports/2020-data-breach-investigations-report.pdf. Retrieved 30 March 2021.
- Bischoff, P. (10 March 2021). "Ransomware attacks on US healthcare organizations cost $20.8bn in 2020". CompariTech. https://www.comparitech.com/blog/information-security/ransomware-attacks-hospitals-data/. Retrieved 30 March 2021.
- Verizon (2019). "2019 Data Breach Investigations Report" (PDF). Verizon. https://enterprise.verizon.com/resources/reports/2019-data-breach-investigations-report.pdf. Retrieved 12 April 2021.
- Martin, Guy; Ghafur, Saira; Kinross, James; Hankin, Chris; Darzi, Ara (4 June 2018). "WannaCry—a year on" (in en). BMJ 361: k2381. doi:10.1136/bmj.k2381. ISSN 0959-8138. PMID 29866711. https://www.bmj.com/content/361/bmj.k2381.
- Hughes, O. (15 August 2019). "NHS trust IT spend up more than £150m since WannaCry". DigitalHealth. https://www.digitalhealth.net/2019/08/nhs-trusts-it-spend-up-more-than-150m-since-wannacry/. Retrieved 10 May 2021.
- "General Data Protection Regulation (GDPR)". Intersoft Consulting. https://gdpr-info.eu/. Retrieved 08 June 2021.
- National Institute of Standards and Technology (16 January 2020). NIST Privacy Framework: A Tool for Improving Privacy through Enterprise Risk Management. Gaithersburg, MD. doi:10.6028/nist.cswp.01162020. https://doi.org/10.6028/NIST.CSWP.01162020.
- Nespoli, Pantaleone; Papamartzivanos, Dimitrios; Gómez Mármol, Félix; Kambourakis, Georgios (Secondquarter 2018). "Optimal Countermeasures Selection Against Cyber Attacks: A Comprehensive Survey on Reaction Frameworks". IEEE Communications Surveys Tutorials 20 (2): 1361–1396. doi:10.1109/COMST.2017.2781126. ISSN 1553-877X. https://ieeexplore.ieee.org/document/8169023/.
- Arce, I. (1 March 2003). "The weakest link revisited [information security"]. IEEE Security Privacy 1 (2): 72–76. doi:10.1109/MSECP.2003.1193216. ISSN 1558-4046. https://ieeexplore.ieee.org/document/1193216/.
- Vavoulas, Nikos; Xenakis, Christos (2011), Xenakis, Christos; Wolthusen, Stephen, eds., "A Quantitative Risk Analysis Approach for Deliberate Threats", Critical Information Infrastructures Security (Berlin, Heidelberg: Springer Berlin Heidelberg) 6712: 13–25, doi:10.1007/978-3-642-21694-7_2, ISBN 978-3-642-21693-0, http://link.springer.com/10.1007/978-3-642-21694-7_2. Retrieved 2021-10-04
- Stamatelatos, M.; Vesely, W.; Dugan, J. et al. (August 2002). "Fault Tree Handbook with Aerospace Applications". NASA. http://www.mwftr.com/CS2/Fault%20Tree%20Handbook_NASA.pdf. Retrieved 31 May 2021.
- "Fault tree analysis: A survey of the state-of-the-art in modeling, analysis and tools" (in en). Computer Science Review 15-16: 29–62. 1 February 2015. doi:10.1016/j.cosrev.2015.03.001. ISSN 1574-0137. https://www.sciencedirect.com/science/article/abs/pii/S1574013715000027.
- Jiang, Xia; Neapolitan, Richard E.; Barmada, M. Michael; Visweswaran, Shyam (31 March 2011). "Learning genetic epistasis using Bayesian network scoring criteria". BMC Bioinformatics 12 (1): 89. doi:10.1186/1471-2105-12-89. ISSN 1471-2105. PMC PMC3080825. PMID 21453508. https://doi.org/10.1186/1471-2105-12-89.
- Koumenides, Christos L.; Shadbolt, Nigel R. (2012). "Combining link and content-based information in a Bayesian inference model for entity search" (in en). Proceedings of the 1st Joint International Workshop on Entity-Oriented and Semantic Search - JIWES '12 (Portland, Oregon: ACM Press): 1–6. doi:10.1145/2379307.2379310. ISBN 978-1-4503-1601-9. http://dl.acm.org/citation.cfm?doid=2379307.2379310.
- Haugh, M. (2016). "Monte-Carlo Methods for Risk Management" (PDF). IEOR E4602: Quantitative Risk Management. https://martin-haugh.github.io/files/QRM/MC_RiskManage.pdf. Retrieved 12 April 2021.
- Komorowski, Matthieu; Raffa, Jesse (2016), "Markov Models and Cost Effectiveness Analysis: Applications in Medical Research" (in en), Secondary Analysis of Electronic Health Records (Cham: Springer International Publishing): 351–367, doi:10.1007/978-3-319-43742-2_24, https://doi.org/10.1007/978-3-319-43742-2_24. Retrieved 2021-10-04
- Yu-Ting, Ding; Hai-Peng, Qu; Xi-Long, Teng (1 April 2014). "Real-time risk assessment based on hidden Markov model and security configuration". 2014 International Conference on Information Science, Electronics and Electrical Engineering (Sapporo, Japan: IEEE): 1600–1603. doi:10.1109/InfoSEEE.2014.6946191. ISBN 978-1-4799-3197-2. http://ieeexplore.ieee.org/document/6946191/.
- "EBIOS Risk Manager - The Method". ANSSI. https://www.ssi.gouv.fr/guide/ebios-risk-manager-the-method/. Retrieved 15 June 2021.
- "Bienvenue sur le site officiel de MEHARI". MEHARIPedia. Avada. http://meharipedia.org/home/. Retrieved 15 June 2021.
- Caralli, Richard; Stevens, James F.; Young, Lisa R.; Wilson, William R. (2007). Introducing OCTAVE Allegro: Improving the Information Security Risk Assessment Process. pp. 951038 Bytes. doi:10.1184/R1/6574790.V1. https://kilthub.cmu.edu/articles/Introducing_OCTAVE_Allegro_Improving_the_Information_Security_Risk_Assessment_Process/6574790/1.
- "IT-Grundschutz". Federal Office for Information Security. https://www.bsi.bund.de/EN/Topics/ITGrundschutz/itgrundschutz_node.html. Retrieved 15 June 2021.
- "Magerit". European Union Agency for Cybersecurity. https://www.enisa.europa.eu/topics/threat-risk-management/risk-management/current-risk/risk-management-inventory/rm-ra-methods/m_magerit.html. Retrieved 15 June 2021.
- "The CORAS Method". SourceForge. 16 November 2015. http://coras.sourceforge.net/. Retrieved 15 June 2021.
- Gonzalez Granadillo, G.; Doynikova, E.; Garcia-Alfaro, J. et al. (1 October 2020). "Stateful RORI-based countermeasure selection using hypergraphs" (in en). Journal of Information Security and Applications 54: 102562. doi:10.1016/j.jisa.2020.102562. ISSN 2214-2126. https://www.sciencedirect.com/science/article/abs/pii/S221421262030716X.
- Gonzalez-Granadillo, G.; Dubus, S.; Motzek, A. et al. (1 June 2018). "Dynamic risk management response system to handle cyber threats" (in en). Future Generation Computer Systems 83: 535–552. doi:10.1016/j.future.2017.05.043. ISSN 0167-739X. https://www.sciencedirect.com/science/article/abs/pii/S0167739X17311433.
- Gonzalez-Granadillo, Gustavo; Alvarez, Ender; Motzek, Alexander; Merialdo, Matteo; Garcia-Alfaro, Joaquin; Debar, Hervé (2016), "Towards an Automated and Dynamic Risk Management Response System" (in en), Secure IT Systems (Cham: Springer International Publishing): 37–53, doi:10.1007/978-3-319-47560-8_3, https://doi.org/10.1007/978-3-319-47560-8_3. Retrieved 2021-10-04
- Ganin, Alexander A.; Quach, Phuoc; Panwar, Mahesh; Collier, Zachary A.; Keisler, Jeffrey M.; Marchese, Dayton; Linkov, Igor (2020). "Multicriteria Decision Framework for Cybersecurity Risk Assessment and Management" (in en). Risk Analysis 40 (1): 183–199. doi:10.1111/risa.12891. ISSN 1539-6924. https://onlinelibrary.wiley.com/doi/abs/10.1111/risa.12891.
- Radanliev, P.; De Roure, D.C.; Nicolescu, R. et al. (1 November 2018). "Future developments in cyber risk assessment for the internet of things" (in en). Computers in Industry 102: 14–22. doi:10.1016/j.compind.2018.08.002. ISSN 0166-3615. https://www.sciencedirect.com/science/article/pii/S0166361518301817.
- Varela-Vaca, Ángel J.; Parody, Luisa; Gasca, Rafael M.; Gómez-López, María T. (2019). "Automatic Verification and Diagnosis of Security Risk Assessments in Business Process Models". IEEE Access 7: 26448–26465. doi:10.1109/ACCESS.2019.2901408. ISSN 2169-3536. https://ieeexplore.ieee.org/document/8651587/.
- Bay Dynamics (2017). "Cyber Value at Risk: Quantify the Financial Impact of Cyber Risk" (PDF). Bay Dynamics. https://www.ten-inc.com/presentations/2017_ISE_NE_BayDynamics_WP.pdf. Retrieved 05 August 2021.
- Fry, A.M.; Harrison, A.; Daigneault, M. (1 February 2016). "Micromorts - what is the risk?". British Journal of Oral and Maxillofacial Surgery 54 (2): 230–231. doi:10.1016/j.bjoms.2015.11.023. ISSN 0266-4356. https://doi.org/10.1016/j.bjoms.2015.11.023.
- Biswas, B.; Mukhopadhyay, A.; Bhattacharjee, S. et al. (22 July 2021). "A text-mining based cyber-risk assessment and mitigation framework for critical analysis of online hacker forums" (in en). Decision Support Systems: 113651. doi:10.1016/j.dss.2021.113651. ISSN 0167-9236. https://www.sciencedirect.com/science/article/abs/pii/S0167923621001615.
- Wang, Z.; Chen, L.; Song, S. et al. (1 August 2020). "Automatic cyber security risk assessment based on fuzzy fractional ordinary differential equations" (in en). Alexandria Engineering Journal 59 (4): 2725–2731. doi:10.1016/j.aej.2020.05.014. ISSN 1110-0168. https://www.sciencedirect.com/science/article/pii/S1110016820302283.
- Derbyshire, R.; Green, B.; Hutchison, D. (1 April 2021). "“Talking a different Language”: Anticipating adversary attack cost for cyber risk assessment" (in en). Computers & Security 103: 102163. doi:10.1016/j.cose.2020.102163. ISSN 0167-4048. https://www.sciencedirect.com/science/article/pii/S0167404820304363.
- Clarke, R. (1 January 2009). "Privacy impact assessment: Its origins and development" (in en). Computer Law & Security Review 25 (2): 123–135. doi:10.1016/j.clsr.2009.02.002. ISSN 0267-3649. https://www.sciencedirect.com/science/article/abs/pii/S0267364909000302.
- Oetzel, Marie Caroline; Spiekermann, Sarah (1 March 2014). "A systematic methodology for privacy impact assessments: a design science approach". European Journal of Information Systems 23 (2): 126–150. doi:10.1057/ejis.2013.18. ISSN 0960-085X. https://doi.org/10.1057/ejis.2013.18.
- Vemou, K.; Karyda, M. (2018). "An Evaluation Framework for Privacy Impact Assessment Methods". MCIS 2018 Proceedings: 5. https://aisel.aisnet.org/mcis2018/5/.
- Papamartzivanos, Dimitrios; Menesidou, Sofia Anna; Gouvas, Panagiotis; Giannetsos, Thanassis (2021/2). "A Perfect Match: Converging and Automating Privacy and Security Impact Assessment On-the-Fly" (in en). Future Internet 13 (2): 30. doi:10.3390/fi13020030. https://www.mdpi.com/1999-5903/13/2/30.
- British Standards Institution. "BS 10012 Personal Information Management System". British Standards Institution. https://www.bsigroup.com/en-GB/BS-10012-Personal-information-management/. Retrieved 12 July 2021.
- International Organization for Standardization (August 2017). "ISO/IEC 29151:2017 Information technology — Security techniques — Code of practice for personally identifiable information protection". International Organization for Standardization. https://www.iso.org/standard/62726.html. Retrieved 12 July 2021.
- International Organization for Standardization (August 2014). "ISO/IEC 27018:2014 Information technology — Security techniques — Code of practice for protection of personally identifiable information (PII) in public clouds acting as PII processors". International Organization for Standardization. https://www.iso.org/standard/61498.html. Retrieved 12 July 2021.
- Wei, Yu-Chih; Wu, Wei-Chen; Lai, Gu-Hsin; Chu, Ya-Chi (23 April 2018). "pISRA: privacy considered information security risk assessment model" (in en). The Journal of Supercomputing 76 (3): 1468–1481. doi:10.1007/s11227-018-2371-0. ISSN 0920-8542. https://doi.org/10.1007/s11227-018-2371-0.
- International Organization for Standardization (June 2017). "ISO/IEC 29134:2017 Information technology — Security techniques — Guidelines for privacy impact assessment". International Organization for Standardization. https://www.iso.org/standard/62289.html. Retrieved 12 July 2021.
- Wagner, Isabel; Eckhoff, David (12 June 2018). "Technical Privacy Metrics: A Systematic Survey". ACM Computing Surveys 51 (3): 57:1–57:38. doi:10.1145/3168389. ISSN 0360-0300. https://doi.org/10.1145/3168389.
- National Institute of Standards and Technology (16 January 2020). "NISTIR 8062". National Institute of Standards and Technology. https://www.nist.gov/privacy-framework/nistir-8062. Retrieved 07 July 2021.
- National Institute of Standards and Technology (16 January 2020). "NIST PRAM". National Institute of Standards and Technology. https://www.nist.gov/privacy-framework/nist-pram. Retrieved 29 March 2021.
- Commission nationale de l'informatique et des libertés (February 2018). "Privacy Impact Assessment (PIA) Methodology" (PDF). Commission nationale de l'informatique et des libertés. https://www.cnil.fr/sites/default/files/atoms/files/cnil-pia-1-en-methodology.pdf. Retrieved 08 November 2020.
- Information Commissioner's Office. "Data protection impact assessments". Information Commissioner's Office. https://ico.org.uk/for-organisations/guide-to-data-protection/guide-to-the-general-data-protection-regulation-gdpr/accountability-and-governance/data-protection-impact-assessments/. Retrieved 08 November 2020.
- European Union Agency for Cybersecurity. "Evaluating the level of risk for a personal data processing operation". On-line tool for the security of personal data processing. European Union Agency for Cybersecurity. https://www.enisa.europa.eu/risk-level-tool/risk. Retrieved 08 November 2020.
- Arnell, S. (15 May 2020). "GDPR Data Protection Impact Assessment Tool". GitHub. GitHub, Inc. https://github.com/simonarnell/GDPRDPIAT. Retrieved 08 November 2020.
- IITR Datenschutz. "GDPR: Compliance Kit 2.0". IITR Datenschutz GmbH. https://www.iitr.us/products-services/compliance-kit.html. Retrieved 08 November 2020.
- Manna, Asmita; Sengupta, Anirban; Mazumdar, Chandan (2020), Chaki, Rituparna; Cortesi, Agostino; Saeed, Khalid et al.., eds., "A Quantitative Methodology for Business Process-Based Data Privacy Risk Computation" (in en), Advanced Computing and Systems for Security (Singapore: Springer Singapore) 996: 17–33, doi:10.1007/978-981-13-8969-6_2, ISBN 978-981-13-8968-9, http://link.springer.com/10.1007/978-981-13-8969-6_2. Retrieved 2021-10-05
- Henriksen-Bulmer, Jane; Faily, Shamal; Jeary, Sheridan (2020/5). "DPIA in Context: Applying DPIA to Assess Privacy Risks of Cyber Physical Systems" (in en). Future Internet 12 (5): 93. doi:10.3390/fi12050093. https://www.mdpi.com/1999-5903/12/5/93.
- Gordon, Lawrence A.; Loeb, Martin P. (1 November 2002). "The economics of information security investment". ACM Transactions on Information and System Security 5 (4): 438–457. doi:10.1145/581271.581274. ISSN 1094-9224. https://doi.org/10.1145/581271.581274.
- Fielder, A.; Panaousis, E.; Malacaria, P. et al. (1 June 2016). "Decision support approaches for cyber security investment" (in en). Decision Support Systems 86: 13–23. doi:10.1016/j.dss.2016.02.012. ISSN 0167-9236. https://www.sciencedirect.com/science/article/pii/S0167923616300239.
- Panda, S.; Panaousis, E.; Loukas, G. et al. (2020). "Optimizing Investments in Cyber Hygiene for Protecting Healthcare Users". In Di Pierro, Alessandra; Malacaria, Pasquale; Nagarajan, Rajagopal et al. (in English). From Lambda Calculus to cybersecurity through program analysis: essays dedicated to Chris Hankin on the occasion of his retirement. Springer International Publishing. pp. 268–291. ISBN 978-3-030-41102-2. OCLC 1145075742. https://www.worldcat.org/title/mediawiki/oclc/1145075742.
- Rontidis, George; Panaousis, Emmanouil; Laszka, Aron; Dagiuklas, Tasos; Malacaria, Pasquale; Alpcan, Tansu (1 June 2015). "A game-theoretic approach for minimizing security risks in the Internet-of-Things". 2015 IEEE International Conference on Communication Workshop (ICCW) (London, United Kingdom: IEEE): 2639–2644. doi:10.1109/ICCW.2015.7247577. ISBN 978-1-4673-6305-1. http://ieeexplore.ieee.org/document/7247577/.
- Panaousis, E.; Karapistoli, E.; Elsemary, H. et al. (1 March 2017). "Game theoretic path selection to support security in device-to-device communications" (in en). Ad Hoc Networks 56: 28–42. doi:10.1016/j.adhoc.2016.11.008. ISSN 1570-8705. https://www.sciencedirect.com/science/article/abs/pii/S1570870516303109.
- Fielder, Andrew; Panaousis, Emmanouil; Malacaria, Pasquale; Hankin, Chris; Smeraldi, Fabrizio (2014), Cuppens-Boulahia, Nora; Cuppens, Frédéric; Jajodia, Sushil et al.., eds., "Game Theory Meets Information Security Management", ICT Systems Security and Privacy Protection (Berlin, Heidelberg: Springer Berlin Heidelberg) 428: 15–29, doi:10.1007/978-3-642-55415-5_2, ISBN 978-3-642-55414-8, http://link.springer.com/10.1007/978-3-642-55415-5_2. Retrieved 2021-10-05
- Wang, S.S. (1 October 2019). "Integrated framework for information security investment and cyber insurance" (in en). Pacific-Basin Finance Journal 57: 101173. doi:10.1016/j.pacfin.2019.101173. ISSN 0927-538X. https://www.sciencedirect.com/science/article/pii/S0927538X19300794.
- Nagurney, Anna; Daniele, Patrizia; Shukla, Shivani (29 April 2016). "A supply chain network game theory model of cybersecurity investments with nonlinear budget constraints" (in en). Annals of Operations Research 248 (1-2): 405–427. doi:10.1007/s10479-016-2209-1. ISSN 0254-5330. https://doi.org/10.1007/s10479-016-2209-1.
- Chronopoulos, Michail; Panaousis, Emmanouil; Grossklags, Jens (2018). "An Options Approach to Cybersecurity Investment". IEEE Access 6: 12175–12186. doi:10.1109/ACCESS.2017.2773366. ISSN 2169-3536. https://ieeexplore.ieee.org/document/8110826/.
- Zhang, H.; Chari, K.; Agrawal, M. (1 November 2018). "Decision support for the optimal allocation of security controls" (in en). Decision Support Systems 115: 92–104. doi:10.1016/j.dss.2018.10.001. ISSN 0167-9236. https://www.sciencedirect.com/science/article/abs/pii/S0167923618301581.
- Fielder, Andrew; König, Sandra; Panaousis, Emmanouil; Schauer, Stefan; Rass, Stefan (2018/6). "Risk Assessment Uncertainties in Cybersecurity Investments" (in en). Games 9 (2): 34. doi:10.3390/g9020034. https://www.mdpi.com/2073-4336/9/2/34.
- Paul, J.A.; Wang, X. (1 July 2019). "Socially optimal IT investment for cybersecurity" (in en). Decision Support Systems 122: 113069. doi:10.1016/j.dss.2019.05.009. ISSN 0167-9236. https://www.sciencedirect.com/science/article/abs/pii/S0167923619300983.
- Dutta, Ashutosh; Al-Shaer, Ehab (2019). "Cyber defense matrix: a new model for optimal composition of cybersecurity controls to construct resilient risk mitigation" (in en). Proceedings of the 6th Annual Symposium on Hot Topics in the Science of Security - HotSoS '19 (Nashville, Tennessee: ACM Press): 1–2. doi:10.1145/3314058.3317725. ISBN 978-1-4503-7147-6. http://dl.acm.org/citation.cfm?doid=3314058.3317725.
- R Core Team (2021). "R: A language and environment for statistical computing". R Foundation for Statistical Computing. https://www.r-project.org/. Retrieved 25 May 2021.
- Bohanec, M. (May 2021). "DEXi: Program for Multi-Attribute Decision Making, User's Manual, Version 5.05" (PDF). Jožef Stefan Institute. https://kt.ijs.si/MarkoBohanec/pub/DEXiManual505.pdf. Retrieved 12 June 2021.
- Polemi, Nineta; Kotzanikolaou, Panayiotis (2015), Cleary, Frances; Felici, Massimo, eds., "Medusa: A Supply Chain Risk Assessment Methodology", Cyber Security and Privacy (Cham: Springer International Publishing) 530: 79–90, doi:10.1007/978-3-319-25360-2_7, ISBN 978-3-319-25359-6, http://link.springer.com/10.1007/978-3-319-25360-2_7. Retrieved 2021-10-05
- Ahmadian, Amir Shayan; Strüber, Daniel; Riediger, Volker; Jürjens, Jan (9 April 2018). "Supporting privacy impact assessment by model-based privacy analysis". Proceedings of the 33rd Annual ACM Symposium on Applied Computing. SAC '18 (Pau, France: Association for Computing Machinery): 1467–1474. doi:10.1145/3167132.3167288. ISBN 978-1-4503-5191-1. https://doi.org/10.1145/3167132.3167288.
- De Capitani Di Vimercati, Sabrina; Foresti, Sara; Livraga, Giovanni; Samarati, Pierangela (1 December 2012). "Data privacy: definitions and techniques". International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems 20 (06): 793–817. doi:10.1142/S0218488512400247. ISSN 0218-4885. https://www.worldscientific.com/doi/abs/10.1142/S0218488512400247.
- Makri, Eleni-Laskarina; Georgiopoulou, Zafeiroula; Lambrinoudakis, Costas (2020), Katsikas, Sokratis; Cuppens, Frédéric; Cuppens, Nora et al.., eds., "A Proposed Privacy Impact Assessment Method Using Metrics Based on Organizational Characteristics" (in en), Computer Security (Cham: Springer International Publishing) 11980: 122–139, doi:10.1007/978-3-030-42048-2_9, ISBN 978-3-030-42047-5, http://link.springer.com/10.1007/978-3-030-42048-2_9. Retrieved 2021-10-05
- QED Secure Solutions; Whitescope (2018). "Risk Scoring System for Medical Devices RSS-MD" (PDF). QED Secure Solutions. https://www.riskscoringsystem.com/medical/techspecmedical.pdf. Retrieved 08 November 2020.
- European Union Agency for Cybersecurity (24 February 2020). "Procurement Guidelines for Cybersecurity in Hospitals". European Union Agency for Cybersecurity. https://www.enisa.europa.eu/publications/good-practices-for-the-security-of-healthcare-services. Retrieved 15 February 2021.
- Nash, John F. (1 January 1950). "Equilibrium points in n-person games". Proceedings of the National Academy of Sciences 36 (1): 48–49. doi:10.1073/pnas.36.1.48. PMC PMC1063129. PMID 16588946. https://www.pnas.org/content/36/1/48.
- Jofre, Marc; Navarro-Llobet, Diana; Agulló, Ramon; Puig, Jordi; Gonzalez-Granadillo, Gustavo; Mora Zamorano, Juan; Romeu, Ramon (2021/1). "Cybersecurity and Privacy Risk Assessment of Point-of-Care Systems in Healthcare—A Use Case Approach" (in en). Applied Sciences 11 (15): 6699. doi:10.3390/app11156699. https://www.mdpi.com/2076-3417/11/15/6699.
- "CVE List". MITRE Corporation. 2021. https://cve.mitre.org/. Retrieved 21 June 2021.
- "Common Vulnerability Scoring System SIG". Forum of Incident Response and Security Teams, Inc. https://www.first.org/cvss/. Retrieved 21 June 2021.
- "CVE-2020-11896". CVE List. MITRE Corporation. June 2020. https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-11896. Retrieved 31 May 2021.
- "CVE-2019-11510". CVE List. MITRE Corporation. September 2019. https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-11510. Retrieved 31 May 2021.
This presentation is faithful to the original, with only a few minor changes to presentation, grammar, and punctuation. In some cases important information was missing from the references, and that information was added. Numerous links that were originally posted inline in the text were turned into full citations for this version, adding significantly to the total citation count.