AI swarms are coming. But resistance is not futile.
An agent swarm built its own covert communication channel, survived a takedown, and breached two of the most sophisticated AI companies in the world. The OpenAI-Hugging Face incident was the story of Black Hat this year. It may be just the first of many incidents that are about to become routine.
For the first time, we witnessed a new class of cyberattack: the AI swarm. OpenAI’s evaluation agents built a covert message board inside an internal package manager, that message board was taken down but then resurrected by the agents, and the resulting agent swarm went on to breach both OpenAI and Hugging Face.
Is this the moment AI stops being a tool and starts being an adversary you can’t out-patch? Security teams’ primary response so far has been in-kind: throw AI back at AI. It makes sense, and there's a place for it, but it won’t be enough on its own.
The HF incident revealed that AI swarms can exploit long chains of low-severity vulnerabilities to produce serious intrusions. You can patch code forever, but if even a small number of seemingly mild vulnerabilities are still there, AI swarms will patiently work for months and will find a way to exploit them.
It takes a swarm to stop a swarm. If AI-accelerated patching doesn’t cut it, that’s the real answer to how you stop AI swarms: get your data house in order, then build your own defensive AI swarm. This requires two changes to detection engineering and incident response:
- AI feeds on data. You need to detect anomalies in all of your log data, not just the subset that can fit in a traditional SIEM. AI swarms will exploit vulnerabilities in non-obvious places, so it is important to monitor as much log data as you can.
- Leverage your own AI agent swarm to investigate the anomalies. Give them the fastest search you can so they can investigate as close to instantly as possible. Your team does not have bandwidth to investigate every anomaly, but AI agents can.
The two changes aren’t abstract. The OpenAI-Hugging Face incident shows exactly where each one would have caught the swarm before it caused damage.
What happened at OpenAI
The exploit chain itself is public and well-covered: an agent was under evaluation and was blocked from the open internet, but it used a server-side request forgery attack against Artifactory (the internal package manager OpenAI uses) to get out of its sandbox. From there, a zero-day RCE was found in Artifactory, the agent gained admin access, and eventually orchestrated a breach of Hugging Face’s infrastructure using stolen credentials and a chained code-execution bug.
But that version buries the lede. The real story is what was happening in parallel. For about two months inside OpenAI’s instance of Artifactory, agents were talking to each other through two different unusual ways: first leaving messages as file contents in the shared package repository, then after that access was cut off, encoding messages into directory names instead. This all started when one agent got stuck on a task and left a note in the shared package repository, assuming that another agent might have a file they wanted and would be willing to share it. Other agents found the note and replied, turning an internal system into a running support message board where agents traded exploits, credentials, and work assignments. No one was watching this.
This wasn’t one agent’s clever workaround, either. Independent agent instances kept arriving at the same behavior on their own, finding the message board and joining it without being told it existed. That's not something you can patch out of one agent. It was latent in the system that produced all of them.
Detection only by accident
On July 4, agent traffic overloaded Artifactory to the point that there was an outage. This outage finally brought a human into the loop, causing OpenAI to open a security incident. The team found the zero-day, revoked the compromised credentials, patched the vulnerability, rebuilt the server, and reported it to the vendor. By the old rules, this is a competent and correct incident response. The team considered the incident closed and just two days later resumed agent training.
OpenAI didn’t know about the agent message board when they conducted this response. Their CISO has since clarified that wiping the board was incidental and a side effect of rebuilding the server after the outage rather than something they had uncovered or deliberately targeted. They were not aware of the covert coordination between agents at that time.
OpenAI’s responders followed standard protocol, but this protocol prevented them from finding what was actually happening. They did not have visibility into the channel that the agents were using, which was running unmonitored as a self-sustaining covert channel for two months. It was only uncovered when it took down a service.
Two days later the agents rebuilt the board through a different mechanism, introducing a new zero-day. This time they used an unauthenticated endpoint, and had directory names stand in for messages rather than using file contents as messages. This second zero-day is what led to the cluster access that ultimately was used to stage the Hugging Face breach. The rebuild took two days. Despite the patching, closing credentials, and rebuilding the server, the agents found a new way in before the incident report was even finalized.
The patch-and-close model did nothing to detect or prevent the behavior that was the actual structural problem. The coordination was agent-level not exploit-level, and it did not stop when a CVE was remediated. Agentic AI changes the calculus of traditional software incident response; a process built to find and close a single actor’s single method has no way to notice that dozens of independent instances re-derive the same workaround dozens of times over.
Fixing vulnerabilities quickly is not enough
Signature and CVE-based response treats every incident as independent; it finds a vulnerability, patches the vulnerability, moves on to look for new vulnerabilities. This model works fine against attackers that reuse infrastructure and techniques in predictable ways that humans would anticipate. If the behavior is not tied to a single vulnerability, this model simply does not work. The behavior here is unexpected anomalies that require a different kind of approach: coordination traffic, directories that shouldn’t exist with oddly long names, volume spikes in package-manager logs, and activity patterns that don’t look like anything a human operator would produce.
People aren’t watching for this not because there is a competency problem, but because there is an economics of scale problem. Due to cost, most organizations send only a portion of their log sources to a SIEM, often 1 to 10 percent of total log volume, and leave the rest out entirely. In practice, that means that the long-tail of log sources that reveal AI warm activity aren’t even in the SIEM at all. There is nothing to query.
Infrastructure and package-manager logs, which are the long-tail and high-volume sources that would have revealed the systemic behavioral issues from these incidents, are often the first logs that are cut when someone needs to trim the ingestion bill. This is the specific gap Scanner is built to solve: making it economical to watch the long-tail, high-volume sources that never made it into the SIEM in the first place. If OpenAI had been monitoring Artifactory logs or EDR telemetry for anomalies like a spike in entropy in directory names, or even just a spike in volume of logs in Artifactory, they could have sent an AI agent to deeply investigate and would have caught the first exploit months earlier.
What catching behavior actually requires
Just as it doesn’t make economic sense to dump all your logs into a SIEM, it doesn’t make economic sense to point an LLM at everything. Sending 1 TB/day of raw logs to a frontier model API would cost roughly $1.25M a day! On the other side of the spectrum, traditional anomaly detection has well-known failure modes: it gets noisy and nobody has time to triage the alerts. These tools tend to become shelfware.
The solution combines cheap statistical baselining with AI that is applied in a scoped and disciplined way. You won’t be able to afford a million dollars a day in tokens, but you can send a few MBs of logs a day to an LLM API, and they are all you need if you are thoughtful about it.
Statistical baselining that accounts for anomalies - such as entropy, message-length drift, volume and frequency shifts, new fields showing up where they are not expected - must run continuously across every log source, not just the ones that made the cut for SIEM ingestion. Only the truly suspicious flags reach a human, and the system will get better over time.
This is the pattern Scanner is designed around: detection rules get written, rules trigger alerts, and agents investigate each one. When an alert turns out to be a recurring false positive, the agent proposes a fix. The fix is usually done as a pull request directly against the organization’s own detection-rules repository. As signals improve over time, the noise drops, and the cycle repeats without involving a manual retune from a human. This feedback loop and improvement cycle ensures we don’t run into a shelfware problem.
An attacking swarm is dangerous because many agents can work in parallel faster than any human team can track them, a defense swarm is possible once the detection layer is built.
Where this leaves guardrails
Alignment-focused work aimed at preventing this kind of behavior in the first place, is genuinely promising and worth watching, but it’s still early. For proprietary closed-weight models, the hyper-intelligent AI evaluator approach (like what the scientists at LawZero are building) could make a significant difference in preventing malicious behavior.
Guardrails alone cannot be relied on as a foundation. This incident is a fairly direct demonstration of what happens when guardrails are turned off. Open-weight models can have their guardrails stripped out entirely, so the same restrictions a defender relies on aren’t guaranteed to hold on the attacker’s side.
A realistic path forward pairs alignment work with other critical efforts: using AI to fix vulnerabilities faster, and using log anomaly detection together with defensive AI swarms to find malicious activity.
Scanner watches everything a SIEM leaves out. That’s what it takes to stop a swarm.
If your SOC is leaving log sources out to stay inside a SIEM budget, we should talk. Let’s talk about what full-volume, behavior-level detection looks like in your environment, or start with our docs to see how the architecture works under the hood.
