SOC noise is the bane of every SOC analyst’s existence. Adding more and more security tools does not solve the problem; in fact, that often makes the alert fatigue issue worse. I've been working on this problem for my entire career in cybersecurity. Ten years ago, I started my first project to try to reduce false positives. Today, I’ve been working with an amazing team of data scientists and security analysts to get network detection technology much, much closer to achieving a better signal-to-noise ratio – one that actually can bring some calm to the chaos.
It is understandable to think that adding a new sensor or a new product will yield better detection coverage, but the infamous side effect is an overload of false positives. An expert system can help weed out some alerts by automating scoring and rank-ordering alerts, so SOC analysts have a better starting point for prioritization. The alert volume, however, can still remain high.
The key to truly cutting out the noise is a network detection and response (NDR) threat engine that can correlate behavior-based threat detections automatically, in turn sending out only events with chained alerts and pre-packaged threat information. That way, the SOC analyst dashboard will highlight only events that require immediate action instead of many random alerts without any situational context or relevance.
Modern data processing systems are based on a hierarchy of sequential processing steps. One-off or single-use analytics can be used for occasional needs, but for embedded, real-time analytics, one must automate a series of successively more sophisticated processing steps, each of which builds on the step that precedes it. For image recognition, this means detecting edges and colors first, then shapes, then objects, and only then classifying the image. For security, this means starting more simply with knowns before proceeding to unknowns.
The knowns of cybersecurity are signatures, or IoCs. Signature-based alerts are table-stakes that also have a relatively low false positive rate. However, they are static and reactive, and therefore have low efficacy for new or evolving threats.
The next level is behavioral detections. These are necessarily based on more data, such as history, statistical measures, and lower-level signatures or “factual” events. Unlike signatures, behavioral detections can identify novel events and new attack types or events that vary by situation or configuration. They result in relatively high false positives, however.
The next frontier in security builds upon the prior levels by automatically correlating signature and behavioral detections. These can yield dynamic, novel, and situational detections, but with relatively low false positives. In addition, since all available relevant information is automatically correlated in advance, these alerts provide actionable content, unlike the simpler events produced by signature and behavioral detection.
Enabled by artificial intelligence (AI), an automated NDR-based correlated threat engine is much more than a collection of computer algorithms. It is informed by human insights. As a result, not only can it automatically pre-correlate alerts across entities but it also can map alerts across the entire kill chain. It also can contextualize the events so the operator doesn't have to spend fruitless time investigating the alerts. Essentially, my team has figured out how to turn human insights into code -- “code-ifying” the timely manual processes needed to dig into an alert to find out more about it before responding. In this way, a threat engine that yields actionable information enables SOC operators to cut to the chase for faster triage.
There are some NDR systems that are beginning to do some form of correlation. An automated correlated threat engine with AI informed by the insights of Tier 3 security experts, however, can look for causal relationships among seemingly random behaviors on the network. For instance, let’s say for one entity, potential spearfishing is detected, as well as a potential beacon. We want to know about the order of those two instances. It is more significant if the spearfishing happened before the beacon than the reverse order. Why? Because a real cyber threat would follow that order, thereby giving more meaning to each of the discrete alerts upon correlating them.
This approach essentially picks up what’s like a “threat language” so that data scientists can encode the logic process or the thinking process that a hunter would use to interrogate events. The difference is that the engine does this automatically before an alert even gets to the human user, cutting through the noise, saving time, and speeding up response.
You can think of this process as a network traffic puzzle that comes with some partially completed sections. You don’t have to waste all that time picking out the edges and corners. You have a clear starting point that makes filling in the rest of the puzzle so much easier. The fact that you can take the middle pieces of a puzzle, correlate those with the endpoint data, and build it across the kill chain is extremely valuable.
As I have said I've been working on this general false-positive problem for a long time. Because my training is in applied mathematics, I originally approached the problem with just a statistical approach. But mathematical optimization can only get you so far. To really solve this complex and dynamic problem, you must incorporate human knowledge and human expertise into your algorithms. As AWS CEO Andy Jassy famously has said, “There is no compression algorithm for experience.”
Leveraging human insights advances the power of analytics, turning the messy reality of SOC detections to more predictable, relevant, and actionable threat alerts.
There is no doubt that increasing visibility by adding network detection capabilities is going to increase the alert fatigue problem, but if you start chaining together alerts in a valid way that's effective for an analyst to triage, you legitimately can reduce the alert fatigue. Think of the automated threat correlation engine as binoculars that let you focus on the two seals sunning on rocks way past the shoreline. In this way, you can fine tune your vast visibility of the threat landscape. A well-tuned threat correlation engine, informed by human insights, is the new way to build a better threat picture – one every SOC operator will be happy to take.