How LLMs are changing threat detection

Threat detection has always suffered from a volume problem. There are always too many logs to manage easily, and many SIEM tools can’t retain enough of them to support historical analysis. When LLMs rose to popularity, many security leaders thought they had a solution. The problem is that the naive use of LLMs for threat detection does little but cost you gobs of money.

The truth is, most security teams have detections but not a detection program. There’s frequently no coverage model, no testing framework, and no improvement loop. In this post, we’ll provide a practical blueprint for building one from the ground up. Once you have a system in place, you can identify the constraints and use LLMs where they actually help. 

Naive use of LLMs won’t improve threat detection, both because of security and efficiency issues. Instead, use LLMs to produce durable assets - detections, tests, or investigation summaries - and keep them away from the raw event stream. Don’t use LLMs to scan every log. 

The (pre-AI) state of threat detection

Back in 2020, Josh Gillner from Apple's detection engineering team gave a talk at Spark Summit about the mechanics of his job and his team’s work. Today, we can use it as a fossilized representation of threat detection before AI arrived. 

According to Gilner, writing, testing, and deploying a single basic detection took his team about a week. New ideas arrived faster than jobs could be deployed, leading to an “unsustainable” pace. Writing tests and preserving test samples made “testing too cumbersome.” More than 60% of new detection code was boilerplate. 

Under these conditions, which were difficult to scale, writing and counting the detections made sense. Detection count was the only number that moved.

Fast forward to 2025, and companies are using LLMs to support threat detection. 

On the drafting side, Google, for example, began using LLMs to generate threat-detection reports meant to ensure executives and stakeholders understand the facts around any given incident. This cut the time engineers spent writing these reports by 53%, at a quality level the team determined was at least comparable.

On the detection side, a team of researchers built GenTI, a benchmark for generating intrusion detection rules against attacks the system has never seen. Detection of those unseen attacks increased from 45% to 87.4%, and the false positive rate dropped from 8.5% to 2.3%, compared with results from baseline LLMs. 

The field is exciting, but it’s not without precedent. It’s not as if threat detection hasn’t changed before, or that automation hasn’t accelerated various steps in the security workflow. 

For example, Zack Allen of Detection Engineering watched a very similar pattern play out with machine learning. Eleven years ago, machine learning provided security teams with incremental time savings that could compound across the organization, saving effort on one end and opening opportunities for better-applied effort on the other. “Our industry has more time to work on harder problems,” he said. “We all became better.”

The catch, he writes, is that "you can't easily verify correctness in ML Security.” Even a black box you designed is, in many circumstances, still a black box. “This is what I think is happening right now with LLMs in our field,” he continues. “The problems are different, the solution is WAY different, but the fundamentals for the most part remain.”

And that’s the key point: The fundamentals remain, even if the problems and solutions are different. Security threats using AI will pose different problems, and security practices enhanced and extended by LLMs will require different skill sets, but the fundamentals–including knowing what good looks like rather than just trusting an LLM’s output–will remain. 

Why you shouldn’t put a model in the event path

We’ve all had moments with LLMs where they felt like magic. How did it know that? How did it do that? It got all that done, with just that prompt? But once you productionize LLMs, the limitations emerge, and as you scale, many of them become hard lines. LLMs, as much as we might wish, shouldn’t be put in the event path. They belong elsewhere. Here’s why.

1. Logs are attacker-controlled input

Security logs record adversarial interaction. That’s the point. 

The HTTP request URI, the user agent, the POST body, the attempted username–all are potential vulnerabilities. A request that probes for SQL injection is, for example, logged by design. If an attacker appends a line to the same request telling the SOC assistant to mark the activity as benign, your evidence stream becomes a dangerous instruction.

Researchers from the University of Houston, PayPal, and Kent State built a benchmark to demonstrate this possibility. LogInject-1.0 contains 12,847 log entries, of which 2,569 are adversarial, spread across Apache access logs, SSH auth logs, and JSON API traces. Tested against GPT-4o, Claude 3.5 Sonnet, and Llama-3-70B, the baseline attack success rate averaged 83.4% and peaked at 88.2%. The researchers call it “passive prompt injection” because the payload sits in storage until an analyst runs a query.

Simon Willison, co-creator of Django, called the underlying dynamic the “lethal trifecta”: private data, untrusted content, and a channel to send things outward. “The problem is that they don’t just follow our instructions,” he writes. “They will happily follow any instructions that make it to the model, whether or not they came from their operator or from some other source.”

You cannot easily filter your way out of this.

2. Log volume creates bottlenecks

Even if you could ensure the security of your logs, you’d still have a volume problem.

Scanner research finds that 87% of enterprise security logs end up unsearchable in cold storage where nobody can search them. This is the kind of coverage gap that, despite its size, often goes unseen and unaccounted for. The smartest teams notice this problem but don’t solve it by pointing LLMs at it and hoping for the best. 

‍Ramp, for example, could retain only 15 days of searchable logs due to budget constraints from its SIEM. Searching through archived logs on Amazon S3 with Amazon Athena was slow, and queries often timed out. 

‍Lemonade, for another example, used a platform with log retention that only included the last 7-30 days, depending on the index. Expensive rehydration fees made any longer-term access too costly. Slow query responses limited how well they could conduct investigations. 

With Scanner, the efficiency changes, and the logic underlying threat detection follows. Ramp now moves about 4 TB of log data a day. Lemonade can now retain logs for more than 12 months while maintaining quick access to historical data. 

LLMs didn’t solve the log volume bottleneck, but once teams address it, they can start using LLMs effectively. That also means avoiding much of the risk that comes from putting LLMs in the event stream in the first place. 

3. Acceleration is risky

Many detection programs are only detection programs in name alone. If a detection program is already ineffective, what does adding an LLM do to it?

The 2026 SANS detection engineering survey found that only 18% of organizations say they are staying ahead of the threats coming at them. The other 80% are “treading water or losing ground,” as the report says. But here’s the contradiction: They report high confidence in their own processes while struggling to keep up. 

The survey also found that vendor-provided rules produce 66% of false positives, which is about flat from the 64% rate the year before. And 43% of teams name cloud-native environments their number one coverage gap, which is more than 2.5 times any other environment.

Bad metrics make this worse. Dave Chismon at the UK National Cyber Centre, for example, argues that the most common SOC metrics tell leadership little about how well the SOC works. Two of the examples he points out are: “Detection rules written” and “Volume of logs collected.” 

Both are the metrics LLMs tend to inflate the fastest. If you grade a team on detection count, and hand them a tool that writes detections in seconds, they will produce an enormous number of detections. And it will look like progress. But as Chismon writes, “Inappropriate metrics can even distract and, in the worst case, harm a SOC's ability to detect and respond to attacks.” 

A blueprint for building a detection program

To build a mature detection program, you don’t need to burn everything down and start over, but you do need to review and reconsider from first principles. Build up layer by layer, and you’ll eventually have a detection program ready to make use of LLMs.

Stage 0: Establish what you can see today

Inventory your telemetry before you write any detections. Score each data source with DeTTECT across:

• Device completeness
• Field completeness
• Timeliness
• Consistency
• Retention. 

Keep visibility scoring separate from detection scoring. 

To better visualize this, take a look at a walkthrough published by SANS. There, Nick Mitropoulos, CEO of Scarlet Dragonfly, writes, “The gap between having a log source and understanding what it actually contains is where most detection programs quietly fall apart.”

Using SSH logs as an example, he writes, “you might be scoring a 2 or 3: limited retention, missing enrichment context, partial field completeness.” Whereas, with EDR logs, “you might be scoring a 4.” The contrast, he writes, “tells you where your detection program is genuinely strong and where it's running on faulty assumptions.”

Here, you should zoom in, scoring specific logs, as in the example above, but you should also zoom out. Treat log retention as a critical program input, not another line item among many. If your searchable window is two weeks, your coverage model has a very limited horizon.

Stage 1: Build a coverage model

The heatmap is where many teams are lying to themselves. For example, researchers analyzed three major endpoint products and found technique coverage ranging from 48% to 55%. Filter out the low- and medium-severity rules, they found, and it drops to about 25%. 

“Combined, our findings highlight the limitations of overdepending on ATT&CK coverage

when evaluating security posture,” the researchers write. “Most notably, many techniques are unrealizable as detection rules, and coverage of an ATT&CK technique does not consistently imply coverage of the same real-world threats.”

That means your coverage model should use ATT&CK but not depend on it entirely. For example, Summiting the Pyramid, now in its fourth version, scores analytics by how hard they are to evade, which turns "Is this a good detection?" into a number you can track over time. 

You can then identify minimum telemetry requirements for ambiguous techniques, and use telemetry confidence scores to rank log sources by relative effectiveness for a given technique. 

Stage 2: Design your detection document

State your goals in plain language, categorize them against ATT&CK, and identify the telemetry the detection depends on. Write down how you validated it, what the false positives look like, and what the responder is supposed to do when it fires. Have someone on your team read the whole thing before any of it becomes a query.

This document-writing stage is one where LLMs can help. Since the text is largely fixed, LLMs can often produce a passable draft, and, as long as humans review before shipping, this step can be accelerated.

Stage 3: Build coverage as code

Store detections in git; ensure changes go through pull requests; and run CI checks on everything. Ramp, for example, made detection-as-code a hard requirement and manages detections in its own GitHub repo with unit testing and change management. FloQast, for another example, deployed coverage via Terraform, wired to GitHub CI/CD.

Stage 4: Test against the environment

Run scripted attacks in a staging environment and measure actual detections against expected detections. 

GitLab's Signals Engineering team built a framework called WATCH that does this, and you can refer to their writeup as a good example. They simulate malicious behavior in staging and verify that the expected alert propagates through the SIEM, out through SOAR routing, and into the dashboards the team watches.

Before each run, WATCH records the detections it expects to trigger with the SOAR, leaving a trackable record of what was supposed to happen. As they write, “tuning false positives is only half the battle.” You need to make sure you test critical detections, too, even if they only fire rarely. 

Keep in mind that this is where you can also test against the attacker-controlled input problem mentioned in the previous section. Add injection payloads to your test corpus and check to ensure your agents don’t act on them.

Stage 5: Deploy and route

Decide who triages, and treat that as a design decision. 

Google, for example, flags the tension between the people writing detections and the people triaging alerts, and argues that splitting them is risky. “When there’s no understanding of the impact of alerts on triage, the incentive to improve detections is lower,” they write.

‍

Slack, for another example, pushed verification out to the source. Their distributed alerting design asks the person who generated the event whether they meant to, and closes the loop with a 2FA confirmation. 

If the answer doesn't come back or the confirmation fails, the security team is notified. 

This is also where you decide what the model is allowed to touch. Alert content includes attacker-written fields, meaning that agents enriching tickets could be reading text from a hostile actor. Treat those fields as data and never as instructions. Give agents a query interface rather than pasting raw log batches into its context, so the text it reads is narrower than the text an attacker controls. 

Stage 6: Monitor, improve, retire

Treat Detection Improvement Requests, scheduled reviews, and decommission decisions as primary outcomes of your detection program. 

You can find a good example in Snowflake, which built a detection development lifecycle that requires the team to review every detection on a schedule and leave it alone, file a Detection Improvement Request, or file a Detection Decommission Request. 

This process, too, is ongoing. As Haider Dost, Snowflake’s Director of Global Threat, Analytics, and Assurance, writes, “Continuous testing of detections is how mature threat detection teams ensure that each detection is accomplishing its intended goal.”

Choosing tooling and metrics

Instead of getting lost in product comparisons pros and cons, focus on the core questions.

To choose the right tooling, ask: 

• Can detections live in version control, with pull requests and CI? 
• Can tests run in the pipeline, including a backtest against real historical events? 
• Can you query enough history, fast enough, so that validation happens before deployment rather than after an incident?
• Does the data stay somewhere you control, and can you leave?

The right platform offers a balance: Tooling that supports engineering best practices without rigid vendor lock-in. 

Metrics require a little more experimentation. To choose the right metrics, try:

• Count of detections with a passing backtest, versus total detections
• Count of detections that have never fired, with an explanation attached to each
• Time from technique publication to a tested detection in production
• Absolute alerts per analyst shift
• Number of decommissions per quarter, which should not be zero

Each metric is itself a mini-experiment. Test them to determine whether they actually measure and improve your detection, or just distract from your company-specific goals. 

Find constraints before adding LLMs

Ineffective systems that run faster with the help of LLMs are still ineffective. If LLMs accelerate work on either side of a system's bottleneck, work piles up instead of clearing.

That's the actual argument of everything above. Visibility scoring, coverage models, detection docs, coverage-as-code, staged testing, deployment routing, ongoing review, each stage exists to tell you where your program is actually weak. Skip that work and point to an LLM at whatever stage feels most painful, and you'll accelerate the wrong thing. A faster path to a bad detection is still a bad detection. A summarized version of an alert nobody understood is still an alert nobody understood.

You don't need to have mastered every stage before an LLM becomes useful. You need to know which stage you're actually stuck at. Fix that constraint first, and the LLM has something real to accelerate. Skip it, and you're just paying to produce more of whatever wasn't working before, faster.