I can remember sometime around late 2001 or early 2002, GREPing Snort logs for that needle in a haystack until I thought I was going to go blind. I further recall around the same time cheering the release of the Analysis Console for Intrusion Databases (ACID) tool which helped to organize the information into something that I could start using to correlate events by way of analysis of traffic patterns.
Skip ahead and the issues we faced while correlating data subtly changed from a one-off analysis to a lack of standardization for the alert formats that were available in the EDR marketplace. Each vendor was producing significant amounts of what was arguably critical information, but unfortunately all in their own proprietary format. This rendered log analysis and information tools constantly behind the 8-ball when trying to ingest all of these critical pieces of disparate event information.
We have since evolved to the point that log file information sharing can be easily facilitated through a number of industry standards, i.e., RFC 6872. Unfortunately, with the advent of the Internet of Things (IoT), we have also created new challenges that must be addressed in order to make the most effective use of data during event correlation. Specifically, how do we quickly correlate and review:
a. Large amounts of data;
b. Data delivered from a number of different resources (IoT);
c. Data which may be trickling in over an extended period of time and,
d. Data segments that, when evaluated separately, will not give insight into the “Big Picture”
How can we now ingest these large amounts of data from disparate devices and rapidly draw conclusions that allow us to make educated decisions during the incident response life cycle? I can envision success coming through the intersection of 4 coordinated activities, all facilitated through event automation:
1. Event filtering – This consists of discarding events that are deemed to be irrelevant by the event correlator. This is also important when we seek to avoid alarm fatigue due to a proliferation of nuisance alarms.
2. Event aggregation – This is a technique where a collection of many similar events (not necessarily identical) are combined into an aggregate that represents the underlying event data.
3. Event Masking – This consists of ignoring events pertaining to systems that are downstream of a failed system.
4. Root cause analysis – This is the last and quite possibly the most complex step of event correlation. Through root cause analysis, we can visualize data juxtapositions to identify similarities or matches between events to detect, determine whether some events can be explained by others, or identify causational factors between security events.
The results of these 4 event activities will promote the identification and correlation of similar cyber security incidents, events and epidemiologies.
According to psychology experts, up to 90% of information is transmitted to the human brain visually. Taking that into consideration, when we are seeking to construct an associational link between large amounts of data we, therefore, must be able to process the information utilizing a visual model. DFLabs IncMan™ provides a feature rich correlation engine that is able to extrapolate information from cyber incidents in order to present the analyst with a contextualized representation of current and historical cyber incident data.
As we can see from the correlation graph above, IncMan has helped simplify and speed up a comprehensive response to identifying the original infection point of entry into the network and then visual representing the network nodes that were subsequently affected, denoted by their associational links.
The ability to ingest large amounts of data and conduct associational link analysis and correlation, while critical, does not have to be overly complicated, provided of course that you have the right tools. If you’re interested in seeing additional capabilities available to simplify your cyber incident response processes, please contact us for a demo at [email protected]