With the U.S. Securities and Exchange Commission requiring CISOs and boards of directors to increase the level of transparency about their organizations’ cybersecurity capabilities and accelerate disclosure of breaches to investors, reporting and cyber metrics have become these This year is an even greater priority for companies.
Boards are giving the reins to security and risk executives to introduce much more rigor into how they monitor key performance indicators (KPIs) and key risk indicators (KRIs) and how they use these metrics to advise and report to the board. Central to both KPIs and KRIs are operational security metrics that track the scope of assets, the cybersecurity activities around those assets, and the security outcomes measured.
“Security teams use operational metrics to track and report on cybersecurity activities and outcomes,” he explains The meeting room of cyber experts, a recent playbook published by a pair of veteran cyber risk leaders to help directors and executive leaders address cyber issues. “When shared with the board’s risk or audit committees, these KPIs highlight the organization’s cybersecurity capabilities and the effectiveness of IT controls, while helping the board evaluate the adequacy of investments in technology and talent.”
Co-authored by Homaira Akbari, CEO of global consultancy AKnowledge Partners, and Shamla Naidoo, head of cloud strategy for Netscope, the book covers a lot of ground, but some of the most important parts of the manual focus on metrics. Dark reading summarizes and excerpts from the tome here to present the most common metrics that Akbari and Naidoo believe are crucial for CISOs to monitor and share with the board of directors in order to report on risk levels and security performance.
The caveat, of course, is that security leaders need to be able to integrate these metrics into easy-to-digest assessments and dashboards. As they explain in their manual, the metrics detailed in each category create a data-driven model to determine the effectiveness of an organization’s program and identify gaps in protection.
“The conclusions of these assessments should be summarized into several overall assessments and included in the company’s cybersecurity dashboard,” they explain.
Data
These metrics should identify risk related to data assets and monitor performance in key protection measures for data security, resilience and continuity. Some of the metrics that Akbari and Naidoo recommend CISOs monitor in this category include:
-
% of employee/customer/user information on the dark web
-
Segmentation depth of the data lake
Financial activities
Risks and losses of financial assets are included in this category: this grouping of parameters should give a measured idea of the financial consequences resulting from recent breaches. Some metrics the authors suggest monitoring (based on the latest quarters or over the past year) include:
-
Value of real money/cryptocurrency lost directly
-
Value for money or productivity losses in the form of ransomware
-
Volume of leaked financial data (accounts, credit cards, loyalty points, online banking credentials)
Although not specifically listed, it would be useful to also monitor data on financial losses resulting from compromised corporate email and the costs of indirectly responding to breaches.
People
Whether falling prey to phishing or business email compromise (BEC) attacks, exposing data by not following policies, or exposing systems in other ways, people typically represent a company’s greatest vulnerability . While it can be difficult to measure the effectiveness of security awareness training, there are some good indicators to get a general idea of the level of adherence of an organization’s staff to security best practices and policies. The authors suggest the following parameters in this category:
-
% of clicks on phishing emails
-
% suspicious emails reported
-
privileged accounts over total accounts
-
% of employees moving data/files out of the company
Other metrics not directly mentioned but still relevant include results from phishing simulations, knowledge assessment scores, and behavioral or account data from high-risk individuals.
Providers
With third-party risk management and digital supply chain security at the forefront of many executives’ minds following events like SolarWinds, boards of directors will want to be informed about the risks of security operations and related performance levels to suppliers. Akbari and Naidoo believe that CISOs would do well to keep the company abreast of trending data and metrics:
-
Self-certification of the cybersecurity posture of third parties
-
External score compared to peers and industry
-
Continuous monitoring of third and fourth posture
-
External audit compliance
-
Penetration test scores (from vendors)
Vendor data will likely overlap heavily with enterprise application metrics (see below), as application security teams begin to consider software supply chain risks, including risky dependencies on third-party code and components.
Infrastructure
Whether on-premises or in the cloud, IT infrastructure exposures and security capabilities in mitigating risks across network and hardware resources should be appropriately monitored and measured. Some operational data that the authors suggest in this category include metrics related to:
-
Number of servers/hardware nearing end of life
-
Secure configurations of all resources
-
Depth of network/infrastructure segmentation
-
Level of inventory automation and hardware asset control
-
Zero Trust Architecture Depth of Implementation: Identity, Device, Access, Services
User-controlled devices
CISOs should be able to give board members an idea of the level of control their organization has over shadow IT and other user-controlled devices operating on the network. Akbari and Naidoo say the following common parameters should be on the radar:
-
Number of unidentified devices on the network
-
Number of devices with unpatched software
-
Number of threats detected and prevented by the endpoint solution
New technologies: IoT
The scale and scope of Internet of Things (IoT) devices have posed significant risks to businesses over the last decade. The authors suggest that CISOs provide some risk metrics regarding these, including:
-
Number of IoT devices that are not upgradeable or patchable
-
Number of IoT ports connecting to corporate networks
-
Depth of segmentation of enterprise resources
While the current focus is on IoT, the same approach could work for all emerging technologies. AI, for example, could include metrics on AI usage and, with some emerging AI security tools, levels of risk exposure resulting from the organization’s use of AI.
Business applications
Whether commercial software or home-grown applications, applications present some of the largest attack surfaces in enterprises today. Akbari and Naidoo offered a couple of common metric cards you should be aware of:
-
Known open software vulnerabilities
-
Software patch pending
-
Number of zero-day software vulnerabilities
There is no shortage additional application security data and metrics which can help monitor performance and risk levels in application portfolios. Consider including data such as the rate of automated versus manual code reviews, time to fix critical vulnerabilities, rate of opening critical vulnerabilities, and metrics that add context about the exploitability or business value of defective assets well-known critics.
Test your safety posture
Security validation and testing are an important part of a security program, and therefore CISOs should be required to track not only the results of security tests, but also the rate at which they conduct the tests. Some parameters that fall into this category, according to Akbari and Naidoo:
-
Penetration test (red, blue).
-
Independent external security ratings from peers and the industry
-
Internal/external auditor’s report on regulatory and IT compliance
-
Scores and findings from the application and other tests
Incident detection and response
Boards of directors will be very interested in a security team’s ability to detect and respond to incidents. Akbari and Naidoo recommend some of the following common operational metrics to track this:
• Volumes and percentage of actual incidents compared to intrusion attempts
• Average time to detect
• Average containment time
• Average time to remedy/resolve
• Red team scores and discoveries
Additionally, CISOs can benefit from offering metrics and results from hands-on exercises and attack simulations if these are what they are engaged in.