Finding a Balance Between Rapid and Measured Incident Response

Since I am a new face (or perhaps just a name to most of you) here at DFLabs, I wanted to take a moment to introduce myself before we jump into the topic for today. My name is John Moran and I recently joined the DFLabs team as Senior Product Manager. Prior to joining the DFLabs team, I worked in a variety of roles, including incident response consulting, security operations and law enforcement. While I have many responsibilities at DFLabs, one of my primary roles and the one that I am perhaps most passionate about is ensuring that DFLabs continues to bring you the industry leading security orchestration, automation and response feature that you have come to expect from IncMan. If you have feature requests, suggestions or other comments, good or bad, regarding IncMan, I’d love to hear from you. Please reach out to me at [email protected]. With that out of the way, let’s get to the good stuff…

While reports such as the Verizon DBIR indicate that the increased focus on creating holistic, detect and respond security programs has had a positive impact on reducing the time to detect security incidents, these same reports have also shown that attackers are continuing to evolve. There is still a continuing gap from compromise to detection. what I would like to discuss here instead though, might be described as the opposite problem; overreaction to a perceived security incident, or conducting a full-scale response to a security incident prior to validating that a security incident has indeed occurred.

Please do not misunderstand what I am saying, I will always advocate the “treat it as an incident until you know otherwise” approach to incident response. However, I would also encourage that the response to any security incident should always be a measured response. The incident response process must be rapid and decisive; but just as under-responding to an incident can present serious financial and reputational risks to an organization, so too can over-responding to a potential security incident. As with any other business process, incident response must provide value to an organization. Continued over-response to perceived security incidents will reduce the overall value that incident response provides to an organization, and over time will result in decreased support from management.

Few studies have truly been able to quantify the costs associated with failing to conduct a measured response. A 2015 study by the Ponemon Institute suggests that response to incidents detected based on erroneous or inaccurate malware alerts costs large organizations up to 395 hours-per-week, or almost $1.3 million a year. It is important to note that this study only took into consideration time spent investigating malware alerts. While malware detection technologies have undoubtedly improved in the two years since this study was conducted, most organizations have a variety of detection technologies, all generating alerts which must be investigated. It was assumed by Ponemon that the organizations surveyed were conducting an appropriate, measured response to each of these false positives. With the cost already so high, it is easy to conclude how costly over-responding to incidents can become at scale.

While conducting incident response consulting, I have personally seen organizations spend weeks to months conducting full-scale incident response activities before spending tens of thousands of dollars for incident response consulting, only to find out that the perceived incident was based on faulty information or conclusions. So how do you minimize the risk of over-responding while continuing to ensure that each potential incident is properly investigated? Here are five tips based on my experience:

  1.  Have the right people in place – There is simply no substitute for having the right people in place. While proper training and experience are vital, the qualities of an effective analyst extend beyond these two attributes. It is crucial to have analysts who possess an analytical mindset and can remain level-headed amidst a stressful and dynamic environment. Training and be provided, the experience can be gained, however, some of these less tangible qualities are much harder to learn.
  2.  Have the right toolsets in place – Attempting to substitute tools for skills will inevitably lead to failure. However, it is important to have the proper tools in place to give those highly skilled analysts the information they need to make fact-based conclusions. Even the most highly skilled analysts will inevitably arrive at the wrong conclusion when presented with incomplete or inaccurate information.
  3.  Know the threat landscape – Threat intelligence, and I mean actual intelligence, not just a machine-readable threat feed, can provide much greater context surrounding a potential security incident. Analysts must also be provided the opportunity to remain up-to-date on the ever-changing threat landscape. This can allow decision makers a much more accurate perspective on which to base their initial level of response. Often, it is a lack of knowledge and conclusions based on assumptions that lead to a dramatic over-response.
  4.  Know your limitations – Unless you are fortunate enough to work for a government agency or one of the world’s largest organization, chances are at some point your needs may exceed the scope of your internal capabilities. These limitations are not weaknesses in and of themselves. Instead, the risk here presents itself when an organization fails to realize its limitation and attempts to work outside of those bounds. It is important to know when to consider tapping into external resources such as consulting, incident response retainers and managed services.
  5.  Replace the emotional response with processes and procedures – Even the most highly skilled analysts will approach some potential security incidents with certain biases or preconceived notions. It is essential to implement quality processes and procedures which maximize the analyst’s skills, take full advantage of the available tools, and guide the incident response process. Processes and procedures surrounding incident validation, incident classification and initial resource allocation can ensure that the process stays on track and avoid straying down the wrong, costly road.

The most important goal of any security program must always remain to never under-respond to an incident. However, integrating these five tips into your security program will undoubtedly provide a better, more efficient process to determine what the appropriate level of response to each potential security incident should be, greatly reducing the risk of over-responding.