Template:Comprehensive Guide to Developing and Implementing a Cybersecurity Plan/Develop and create the cybersecurity plan/Establish performance indicators and associated time frames

5.4 Establish performance indicators and associated time frames

5.4.1 Determine baselines and indicators based on the assessments and objectives from the previous step

Your cybersecurity goals are formulated, their associated objectives are set, and security controls are selected. But how should you best measure their implementation, and over what sort of timeline should they be measured? This is where performance indicators come into play. A performance indicator is "an item of information collected at regular intervals to track the performance of a system." They tend not to be perfect measures of performance, but performance indicators remain an important function of quality control and business management. There's also a social aspect to performance indicators: what is the implied message and behavioral implications of implementing such a monitoring system? Does the monitoring of the indicator, in the end, have a beneficial impact?

Regardless of what industry you work in, deciding on the most appropriate indicators is no easy task. In March 2019, Axio CTO Jason Christopher spoke at a cybersecurity summit about security metrics (a metric is typically a number-based measurement within an indicator), with a focus on the energy industry. During that talk, he discussed various myths concerning collecting metric data for indicators, as well as the mixed success of tools such as heat maps and scorecards. After highlighting the difficulties, he gave a few pieces of useful advice. Among the more interesting suggestions he turned to was a security metrics worksheet to better define, understand, and track what you'll measure for your indicators. In his example, he used the EPRI's (Electric Power Research Institute) Cyber Security Metrics for the Electric Sector document, pulling an example metric and explaining how it was created. Among other aspects, their worksheet format includes an identifier for the metric, the associated organizational goal, and the associated cybersecurity control, which helps ensure the metric is aligned with organizational policy, existing terminology, and current best practices.

Regardless of industry, you may find it useful to use similar worksheet documentation for the indicators you choose to use. Unfortunately, unlike the energy industry, many industries don't have a developed set of technical cybersecurity metrics. However, the ground that EPRI has already covered, plus insights gained during the security controls selection process (see 5.3.10), should aid you in choosing the most appropriate indicators. (An archived version of Jason Christopher's description of the fields on the security metrics worksheet can be found here [PDF]. The EPRI cybersecurity metrics document can be downloaded for free at EPRI.com.) Whatever indicators you choose, be sure they are specific, measurable, actionable, relevant, and focused on a timely nature. In particular, keep the time frame of cybersecurity strategy development and implementation in mind when choosing indicators. If you expect full implementation to take three years but choose indicators outside that time frame, those indicators won't be actionable or timely.

Finally, consider the advice of author and strategic adviser Bernard Marr that business shouldn't be run heavily on performance indicator data. This goes for the development of your indicators for cybersecurity success. Instead, he says, "the focus should be on selecting a robust set of value-adding indicators that serve as the beginning of a rich performance discussion focused on the delivery of your strategy." He continues with a reminder that real people and their actions are behind the indicators, which shouldn't be taken purely at face value.

5.4.2 Determine how to measure progress and assess performance (quantitative vs. qualitative) and what tools are needed for such measurement and assessment

As previously mentioned, with indicators come metrics. But what tools will be used to acquire those metrics, and will those metrics measure quantitatively or qualitatively? Are the measurement and monitoring tools available or will that have to acquired or developed? Can the data from intrusion detection systems and audit logs assist you in developing those metrics? These and other questions must be asked when considering the numbers and measurements associated with an indicator. For many indicators, how to measure progress is relatively clear. A performance indicator such as "mean time to detect" (how long before your business becomes aware of a cybersecurity incident) will be measured in days. An indicator such as "risk classification" (is the risk minor, major, real, etc.) is measured using a non-numerical classification word. Refer to Black et al. and their Cyber security metrics and measures, as well as the HSSEDI (Homeland Security Systems Engineering and Development Institute) document Cyber Risk Metrics Survey, Assessment, and Implementation Plan, for more about cybersecurity metrics.