Noisy security detections
Modern cybersecurity teams are overwhelmed with alerts from a variety of systems such as SIEM, cloud, SaaS application, network, endpoint, etc. Many teams receive hundreds/thousands of alerts every day and large enterprises can receive millions daily. Security analysts are annoyed by repetitive false positives and are extremely concerned about false negatives caused by misjudgment or alerts that are not investigated due to lack of resources. Security analysts are spending more than 50% of their time triaging security alerts every day and noisy security detections are preventing security analysts from making a timely and accurate response to mitigate the risk. Different techniques/technologies have been used to tackle this problem such as fine-tuning detection logic, SOAR (Security Orchestration and Automation), XDR (eXtended Detection and Response), AI (Artificial intelligence), TI (Threat Intelligence). A renowned expert is asking what the better way is to reduce noise in a SOC (security operation center) (link). Tuning the tools better or using SOAR playbooks? Which one do you think?
Before presenting a solution, let’s first look at what are those noisy alerts and where do those noises come from. Security alerts are generated by all types of detections or monitoring tools. Any alerts that are not true positives (true attacks) are adding noises to the overall security signals. On a high level, we can break up the noises into different types.
Type of noises
- Repetitive alerts: alerts that are beeping constantly about the same thing. They could be true positive or false positive alerts.
- Alerts that are clearly false positives and can be determined by simply correlating with internal environment context. For example, a recent update of a certain package generates some abnormal behaviors that look like attackers.
- Alerts that are clearly false positives and can be determined by simply correlating with external threat intelligence. For example, a firewall alert comes from a suspicious IP but was found to be a normal scanner with information from virus total.
- Alerts that are false positives but can only be determined by correlating with other alerts. For example, 100s machines start to generate one type of detection, e.g., a never-seen process starts to run.
- Alerts that are not certain if they are false positives or true positive because they do look like hacker event but has not been seen before.
- Alerts that are not certain if they are false positives, but they do show up in history many times.
A comprehensive solution
Let’s see how to address these different types of noises with a complete solution.
For #1. De-dupe is the way to go. The tricky part lies in how to define “duplicate”. For example, duplicates could be exact same alert or alert that has the same detection type on the same machine. DTonomy provides a flexible way for you to define duplication. (#Dedupe)
For #2. You will need relevant environmental information such as CMDB (configuration management database) information, LDAP etc. to make a judgment. Once you come up with logic, you can automate it to denoise either via fine-tune your detection logic or via the SOAR platform. Either way, you will need to continuously tune it to make it accurate where AI is super valuable here. (#SOAR + #AI)
For #3. For this type of alert, you need external threat intelligence to help make decisions. Once you have logic in place, you will most likely need to rely on the SOAR platform to automate it to denoise. Similarly, you will need to continuously tune it where AI is super valuable here. (#SOAR + #AI + #Threat Intelligence)
For #4. For these types of alerts, AI-based correlation helps reach a conclusion much faster. And once you finalize the logic, you can automate via the SOAR platform or fine-tune the detection rules. Again, it requires continuous fine-tuning where AI is super valuable here. (#SOAR + #AI).
For #5. For these types of alerts, AI helps discover correlated signals and continuously monitors them afterward, and appends them to existing ones if the new context is discovered. This is where XDR kicks in and needs continuous context monitoring. Until you find a logic to justify it is false positives or true positives when you can automate it, monitor it, or spend some time on threat hunting which is typically very time-consuming. We all know you have limited resources. To better utilize your resources, dynamic prioritization with aggregated risk evaluation, including the risk of the new context, the trustworthiness of certain detection, etc., is provided to help prioritize your tasks. This is another place where AI is playing an important role here. (#AI + #SOAR)
For #6. The minimum action to de-noise is to group similar events together with AI. (Aka. pattern discovery), then you can fall back to #5. (#AI)
Too many alerts and too little resources?
The value of AI
You may notice, whether it is fine tuning detection logic or automation, you will still need continuous tuning to make sure your logic is up to date and correct. This is where AI can kick in and play an important role in your triage process besides discovering correlated alerts automatically.
Current detection/automation tunning process:
The current tuning process is usually a trial-and-error process. Here is the standard process
- You propose a filter rule based on your judgment
- You verify through sample data which should cover different types of signals.
- You continuously fine-tune your rules until they satisfy test cases in sample data.
AI offers assistance to simplify the process intelligently:
Continuously validating your detection/automation logic against data is a tedious and risky process. This is where AI kicks in. With assistance from AI, the process will be way easier:
- AI continuously monitors your data and evaluates the detection logic based on your data
- Suggests changes that you should make to your own logic
The value of AI for de-noise:
In summary, in the process of de-noise security alerts, AI is assisting you
- Continuously correlating your incoming threats
- Dynamically prioritizing the aggregated threats
- Providing guidance for fine-tuning your decision logic and optimizing your automation.
Risk of de-noise:
Any actions on de-noise, either fine-tune detection logics or automation, are adding risks of False Negatives (missing attacks). The risk of FN, which is rare, has a potentially unlimited impact. FPs, despite annoying and resource waste, present a limited risk.
De-risk of FNs
- Continuous monitoring and correlation. Even if they are marked as false positives, make sure they are available to be re-correlated once new evidence shows up.
- Do not filter out noise blindly. At least have certain automation in place to make sure they are taken care of by a spare eye – the automated workflow.
DTonomy’s complete solution for de-noise
There is no single silver bullet for solving the noisy detections problems in the cyber security industry. We need all of the technology to work together seamlessly to address this problem that the industry has sought to solve for decades. DTonomy presents a comprehensive solution with multiple well-embedded techniques including AI and SOAR and partners with threat intelligence vendors to help you address this problem.
Try out our platform for free or request a demo for more details.
