Welcome back to my blog series, the CISO’s View. In my last article, CISO’s View: metrics part 1, we started looking at metrics and why they are the foundation of a successful security program. Today, we’ll look at how we derive metrics that communicate value in a way that’s tied to the company strategy. Hopefully by time we are done, you too will agree that metrics are the foundation of a successful security program that is viewed as a strategic business enabler.
Okay, so where do we start? The shift in culture begins with actively participating in helping to develop the company strategy showing how risk management practices both help to achieve the strategic goals and provide an up to date measure on the execution against that strategy. Risk is an interesting word that can have different nuanced meanings. The normal use of risk is the probability and impact that something we care about will suffer loss. This is typically measured using several forms of predictive analytics. For this article, I am using a more general definition of risk as the measure of uncertainty of outcome. This is useful in the context of business strategy as it allows for a formal methodology of setting and measuring execution against business strategy – both the positive and negative effect of uncertainty on that outcome. If your methodology only focuses on loss then you miss the ability to execute on those happy emergent opportunities from positive uncertainty identification. For further reading, check out the OpenFAIR framework to learn more about risk modeling.
Strategy starts with the external inputs and pressures on the company in its drive to compete in the marketplace and provide value to its owners or shareholders. This shouldn’t be anything new, so please allow me to state the obvious. These inputs can take the form of market dominance, profitability, revenue needs, etcetera and become the basis driving corporate strategy. Our job is to be engaged and knowledgeable about these inputs and engage in shaping strategy.
Strategy development is finding the way to meet these external inputs. The focus of this article is how to apply a formal process to these strategies to measure execution and measure the uncertainty in meeting those strategic goals. We need to determine the scenarios that lead to achieving each strategic goal, break down each scenario into the discrete events that lead to a successful outcome, and identify events that could impede or prevent the successful outcome.
Each scenario’s events are then modelled using the predictive risk analytics that determine the points of uncertainty surrounding each event remembering to capture both positive and negative outcomes of uncertainty.
These models get decomposed down to the specific measures in the predictive risk equations and translated into the first level of business metrics expressed as Key Performance Indicators (KPI) and Key Risk Indicators (KRI). These KPIs and KRIs become the basis of tracking execution against strategy and insight into the factors that can impact success.
This is an actual measure of something. There are two main types of performance measures, the positive measure of progress to completion of the specific events that lead to the completion of the goal and the negative measure of the level of effort expended in working on the specific events.
This is the measure of the uncertainty surrounding desired event completion. Again, this is both a predictive analytic score and the degree of measured deviation or uncertainty inherent in the KRI score.
Okay, now that we’ve determine what to measure in a manner that is tied to the company strategy, we need to figure out how to measure. This is identifying the business processes and applications that are involved and determining what data is available to measure against. This should feel familiar, as the data in question is what you’ve probably used to date. The difference is instead of expressing technical metrics such as vulnerability remediation rates, you use the same data to show it applies to the identified KPIs and KRIs.
This is the most overlooked step in a successful metrics driven security program. You’ve done the work figuring out how to communicate your security metrics in a manner that communicates the value your organization provides, and found data that can be used as the source of those metrics. Now what? You’re probably looking at all the work to generate the metrics and either decided it’s not worth the effort, or decided to generate the metrics on some infrequent basis such as quarterly or annually.
Automation is critical
Invest in an enterprise initiative to build a board level reporting dashboard that automates the collection, generation, and reporting of these metrics. Looking back at what we’ve discussed, you’ve probably realized that nothing here is security specific – these same challenges are faced by all IT and business areas. This reporting platform can drive massive efficiency gains – bottom line value – across the enterprise to justify its initial implementation and will continue to show its value in rightsizing spending on what is considered cost center keep-the-lights-on funding.
The goal is to require every new project initiative to consider the data it generates, how that data can be leveraged for metrics, and the method of automated collection into the reporting platform. Every project seeks funding by promising to deliver some value to the business; the reporting platform is a formalized way of actually showing the value delivered.
Okay, that was a bit of a deep dive today on the how behind metrics. Part 3 ties everything together showing how we can communicate to the business and leverage them in a successful security program.
Michael Schiebel is Hortonworks’ GM of Cyber Security Industry where he leverages his over 15 years cybersecurity experience working in financial services and healthcare companies to help customers build cybersecurity analytic solutions. He has lead incident response and computer forensic teams, designed and built security solutions, and created security roadmaps and strategies; learning how to position security projects based on delivering bottom line value to the enterprise.