We get asked a lot of questions about AI. Mostly pertaining to cybersecurity. Mostly. I do get the occasional — “so, when is AI going to take over your job?” But I can confirm that I did in fact write this post. As for the next one…
But security buyers are smart. And considering that enterprise security teams can be managing upwards of 70 security tools in the stack — there are a lot of claims surrounding AI that should have security teams holding vendors accountable.
And being that AI has been at the core of everything we’ve done as a company for over a decade, we’ve compiled a list of questions below that you can use to help get a read on how effective vendors claiming “AI” will be in your stack. And since you asked, yes — it can make your job easier (more on that below), no — you don’t have to feed it (although it does digest information and at an incredibly fast rate), and yes — it makes a great daily companion in the SOC.
1. How do you use AI to detect and stop cyberattacks?
Importance: Regardless of what some vendors might tell you, there is no single algorithm or learning model that will act as silver bullet capable of solving every problem, but rather optimal algorithms for each problem that in cybersecurity generate an attack signal.
You want to uncover how a vendor will help you deal with cyberattacks, and this will come down to signal clarity. Does their attack signal zero in on attacker TTPs post-compromise? How? Will it analyze detection patterns unique to your environment? How? And will it correlate detections across all current and future attack surfaces — network, public cloud, identity, SaaS, etc.?
> Learn how to differenciate AI, Machine Learning and Deep Learning
2. How does your AI-driven attack signal zero in on cyber attacker behaviors?
Importance: By asking this question, you’ll be able to get into the weeds more about how AI will be working behind the scenes in your environment to detect and prioritize attacks. Find out if they have broad coverage for attacker behaviors such as command and control, lateral movement, reconnaissance, etc.
There are some helpful resources that defenders can use to make sure they have coverage for the most current attacker tactics and techniques. One of them being MITRE ATT&CK — a globally accessible knowledge base of adversary methods. This is just one route, but you can ask vendors how their detections map to MITRE ATT&CK. For example, Vectra Attack Signal Intelligence has coverage for over 90% of relevant MITRE ATT&CK techniques — from there, we can show customers how our AI is able to detect a particular method.
You can see how real attacker methods are detected and prioritized in one of our attack anatomies.
Or read about:
> Choosing an Optimal Algorithm in Machine Learning
3. How does your AI prioritize threats targeting high-risk hosts and accounts so analysts know what’s urgent?
Importance: SOC teams receive an average of 4,484 alerts per day. They don’t need more alerts, but rather a way to know which ones matter. The right AI prioritization will help analysts know where to focus their time.
Asking this question will help determine how a vendor’s prioritization model works. You’ll want to know what data sources factor into the equation, which will also help you find out how transparent and forthcoming a vendor is about their algorithms and how they arrive at a threat score. There’s an opportunity to find out how AI correlates threat detections across different attack surfaces, and how it evaluates them to create an urgency rating that an analyst can use to their advantage and ultimately prioritize the most urgent risks.
4. How will your AI reduce the workload for my security analysts?
Importance: Recent Vectra AI research revealed that a majority of SOC analysts say the size of their organization’s attack surface (63%), the number of security tools (70%) and alerts (66%) they manage have significantly increased in the past three years.
When it comes to threat detections, your AI should do more than just provide more threat detections. What are we going to do with the over 4,000 alerts we’re already getting each day — add more alerts? No thanks. Instead, we should be putting AI to work. An AI-driven attack signal can automatically triage and prioritize detections unique to your environment which can greatly reduce detection/alert noise (by as much as 80%) while generating an attack urgency rating so your SOC is armed to do what they do best — stop attacks, not deal with alerts.
5. How will AI help my team investigate and respond to incidents more efficiently?
Importance: Latency is a hybrid attackers’ best friend. Your AI solution should give analysts a head start by sharing context about attacks on prioritized entities with automated and manual options for response.
You already have incident response processes that include people, process, and technology. AI should integrate with your incident response process — where your team currently works. The right AI solution will provide a starting point and guidance for investigations either through its own interface or within existing tools (preferably both) so you have the complete attack narrative to take intelligent action when needed.
6. How will your AI solution integrate with my current security stack?
Importance: Understand how the AI signal will be integrated with your existing security investments. Will you be able to leverage it where your team already operates?
Similar to how vendors should be held accountable for the efficacy of their signal, they should also be held accountable to make sure that every aspect of the implementation process is covered to make it as clear and seamless as possible. Specifically in threat detection and response, your AI solution should provide intelligence to existing infrastructure, and make it easier for your team to respond to urgent threats, while helping maximize current investments.
7. Do you offer AI-driven red team exercises or penetration testing services to further validate your attack signal?
Importance: Real-world testing is critical to validate AI effectiveness. Vendor confidence should extend to covering testing costs if their product underperforms.
Any AI technology used for threat detection and response must understand the TTPs today’s hybrid attackers use. One of the best ways to know for sure if the solution is up to the task, is to simulate real hybrid attacks by emulating both common and sophisticated attack methods. The more accurate the hybrid attack signal, the more SOC analysts know where to focus their time and talent.
8. What can we expect from an AI implementation, analyst experience and investment perspective?
Importance: Talk to vendors about what success looks like for your team in these three areas.
Understand what the learning curve will be for any team members using the tool. Threat detection and response tools, such as XDR, are regularly cited for being overly complicated. An analyst shouldn’t have to work for answers, in fact, it should be the opposite. A good solution is one that gets used, so make sure it will fit into the SOC workflow, which will ultimately help make it a good investment.
9. What SOC workloads can be offloaded to your AI solution?
Importance: According to Gartner, by 2025, 90% of SOCs in the G2000 will use a hybrid model by outsourcing at least 50% of their operational workload.
If more SOC analyst workloads are continuing to expand, what part of their jobs can be trusted to a managed service? We talked about how AI helps SOC teams reduce latency and workloads by producing a high-quality threat signal, but having an option to add managed services on top of that signal can offload tasks like 24x7x365 threat monitoring, detection tuning and triage or other tasks that pull time away from analysts, while also gaining an extension of your team with AI and platform experts.
Does AI translate to an integrated attack signal in your SOC?
AI has certainly come at us full speed over the past couple of years with all types of claims — in many cases from vendors jumping onboard because they can’t afford to not have a stake in ground. But asking the right questions can help clear the path from vendors who will just be another tool in the stack or one that can actually help defenders get ahead of hybrid attack challenges such as latency, workloads and burnout in the SOC. It’s the vendor’s responsibility to deliver the attack signal that will help you stop attacks — we may as well hold them accountable to do so.