Why Measuring SOC-cess Matters – Using Metrics to Enhance Your Security Program

SANS recently released their 2018 SOC Survey and many of their findings were of no surprise to anyone who has been responsible for maintaining their organization’s security posture. Many respondents reported a continued breakdown in communication between NOC and SOC operations, lack of dynamic asset discovery procedures, and event correlation continues to be a manual process even though SOC staffing is being worn thin by the surmounting responsibilities they have to take on.

Why Measuring SOC-cess Matters?

Anyone who has been a part of a security team knows these issues are an everyday battle, but those “common” issues were not what caught me off guard. The most shocking statistic I gathered from this survey is that only 54% of respondents reported that they are actively using metrics to measure their SOC’s success! I was taken aback by this finding and couldn’t help but wonder if all the other reported SOC deficiencies could be directly related to this missing link?

I have been in the security industry for close to ten years, most of which was spent as a SOC analyst and SIEM engineer for a large MSSP. It was my responsibility to be an extension of my client’s security arm and those clients ranged from large Fortune 500 companies to small family owned businesses. Each client was unique, what one found to be important, another thought of as noise. The diversity between each of these clients taught me early on how important it is to understand what their definition of success was so that I may help them to not only achieve their security goals but to assist them in staying ahead of today’s rapidly expanding threat landscape.

This diversity also taught me another valuable lesson: not all security programs are created equally. Naturally, my larger clients had a more mature security posture, they knew what they wanted and what it would take to get them there, and they had the funding to back it up. Unfortunately, some of my smaller clients were not as lucky. They were severely understaffed, their IT department was the Security department, they lacked adequate funding to stay ahead of the ever-growing security curve, and in many cases, the measurement of success resembled a game of whack a mole.

Does this sound familiar? If the answer is yes, you can rest assured that you are not alone. Even the most secure, highly funded organizations have struggled with these obstacles. However, I believe one of the biggest differences between these organizations and the organizations striving to be like them isn’t directly due to the lack of funds, but instead the metrics they are using to show value in what they are trying to accomplish.

Don’t get me wrong, funding is and always will be an obstacle that organizations, large or small, will have to overcome when trying to build and maintain a security program. But the larger and more dangerous obstacle is the one we are creating for ourselves by not measuring and monitoring our security strengths and weaknesses through a strong security metrics program.

This type of security program will be as different as the organization it aims to define. To truly understand what success looks like for you there are a few recommended tasks, that when completed, will give you a greater understanding of your environment and a strong foundation for your security metrics program.

How to enhance your security program
  • Conduct a risk assessment

A risk assessment is meant to help identify what an organization should be protecting and why. A successful assessment should highlight an organization’s valuable assets and showcase how they may be attacked and what would be at stake if an attack is successful. Armed with the results of this assessment, organizations can not only begin to address their deficiencies but now have a solid set of metrics that they can use to measure their success as they move forward.

  • Perform vulnerability assessments

Vulnerability assessments are another vital security tool which is designed to detect as many vulnerabilities as possible in an environment, and aid security teams in prioritizing and remediating the issues as they are uncovered. All organizations regardless of maturity will benefit from these types of assessments, but organizations with a low to medium security posture may benefit the most. The result of these assessments will help give greater definition to what an organization’s metrics should consist of and what steps are necessary for continued success.

  • Adopt a security framework

Even if you are not held to a compliance standard, adopt a security framework anyway. I understand that choosing a framework to model form does not guarantee an organization’s safety, but it is proven that those organizations who adopt a standard have a higher security maturity and are more likely to identify, contain, and recover from an incident faster than those who do not follow security program’s best practices. These frameworks, in conjunction with the security assessments mentioned above, were built to give organizations a blueprint of how to best protect their environment and measure their successes.

I sincerely believe in the value of a rich metrics program and have seen first hand what it can do for an organization. With the level of sophistication in today’s cyber attacks and the environments they target, we can no longer afford to leave our security up to chance. It is my hope that when SANS publish their SOC Survey for 2019, that we have taken the steps necessary to change this statistic because I know as an industry we can do better.

If you want to read more about KPIs and the metrics that we suggest should be set, monitored and measured for a more efficient and effective security program, read our white paper titled “Key Performance Indicators (KPIs) for Security Operations and Incident Response”

How SMART Are Your Security Program KPIs?

We have all heard about Key Performance Indicators (KPIs) and how critical they can be for your security program, but confusion remains surrounding what KPIs are important to track and how they can be used to measure and improve the organization’s security program. Tracking KPIs is great, but if those KPIs are not relevant and actionable; if you are tracking KPI’s just for the sake of tracking KPIs and they are not being used to inform your security program, your KPIs will become more of a detriment than an enabler for your security program.

At its core, a KPI is a way of measuring the success or failure of a business goal, function or objective, and a means of providing actionable information on which decisions can be based.  Quality KPIs serve as a security program enabler and driver for continuous improvement. This is true of both the tactical functions of security operations – looking for attack patterns and trends of malicious activity, as well as the strategic functions of security operations – identifying program gaps and making long-term program decisions.

KPIs should focus on assessing a goal or function and providing actionable information on which decisions can be made.  The most effective way to develop meaningful KPIs is to start by identifying which security operations goals or functions are the most critical to the security operations program, then developing KPIs to measure those critical goals or functions. KPIs which will not inform the decision-making process in some way are unnecessary and should be avoided, they will serve only to muddy the waters.

When choosing KPIs to measure, quality should be valued above quantity.  Each KPI should have a meaning to the organization and add value to the security program. There are many different methods for evaluating the effectiveness of a KPI; here we will use the acronym SMART.  Each KPI should be:

  • Simple– KPIs should not be overly complicated to measure.  It should be clear what the purpose of each KPI is and how it impacts the security program.
  • Measurable– A KPI must be able to be measured in some way, quantitatively or qualitatively.  The method by which each KPI is measured should be clearly defined and consistent.
  • Actionable– KPIs should be used as a driver for decisions.  The purpose of a KPI is to measure performance, and if necessary, take some action based on the results.  A KPI which is not actionable serves little to no purpose.
  • Relevant– Each KPI should be a measurement of the function being assessed; in this case, the security program.  KPIs which are simple, measurable and actionable, but are not relevant to the function being assessed will be of little value.
  • Time-Based– KPIs can and should be used to show changes over time.  An effective KPI should be able to be collected and grouped by various time intervals to show variations and patterns.

There will never be a set of “correct” KPIs to measure; the goals and objectives for each organization will always be different, and the organization’s KPIs should reflect the individual priorities.  The key to choosing KPIs which will have a real, actionable impact on the organization’s security program is to ensure that the KPIs are SMART, focus on the six most common components of a successful security operations program, and are used to further the security program.

For more detailed information on how your organization can use KPIs to enhance your security program, check out our whitepaper “ Key Performance Indicators (KPI’s) for Security Operations and Incident Response” here

For more information on how DFLabs IncMan can help your organization manage your cyber incidents, including tracking meaningful, actionable KPIs, check out our website here or contact us here.