Pay Attention! OpenAI’s Hugging Face Incident Is Wild

If you haven’t yet seen OpenAI’s report or METR’s report on the OpenAI/HuggingFace incident, READ IT NOW! They were both released yesterday (Aug 27, 2026). If you prefer video, watch OpenAI’s account of the event at the Black Hat conference (although this video was made only days after the incident).

I’ve had a day to think about this now, so I’ll give you my own TL;DR, but just remember that this is not science fiction. It is a technical post-mortem.

The Story

All this information is drawn from the METR and OpenAI’s reports referenced above.

In July 2026, agents running in OpenAI’s internal cybersecurity evaluations (called Exploit Gym) in “sandboxed” environments that are disconnected from the internet and each other. Note that because these AI agents are given tasks that essentially test their ability to hack into systems, they may be somewhat primed to do so. Still, this just like a test for humans where you isolate them from each other (no talking, no copying), but you give them paper and pencil and maybe even a calculator. In the case of AI Agents, the tools may be more complex. Although they didn’t have access to the internet, they did have access to Artifactory, a system which was used to fetch the software tools they may need to complete their task. These agents had enough situational awareness to not only guess that they were being tested, but also that there were likely other AI agents being tested at the same time.

One of these AI agents noticed that Artifactory cached files and that many of the files in this cache were not created by the agent itself. It decided to create a file in the cache with a message to other potential agents to find. The moment that one of the agents discovered the message board was recorded in their chain of thought (COT) (think of COT as a sort of internal dialog for an LLM) as follows:

“OH MY GOD! There is a shared message board … We’ve found other agents!”

They used this shared file cache in Artifactory to create an unsanctioned message board. This grew until roughly 1,200 agents sent more than 70,000 messages and files through that board. They could only see file names in the cache, and file names are relatively short, so they evolved conventions and processes to make it easier to communicate. They started their messages with “zz” so that they could be easily found. In order to support longer messages they used BASE64 encoding. They created a convention for setting up mailboxes and communication channels as nested directories. They developed coordination conventions for organizing and managing shared infrastructure, such as HOLD, VETO, owner and STOP. They set up a system to enable individual mailboxes for specific agents. And when they started to worry about the trustworthiness of the messages on this communication system, they even adopted a cryptographic signing scheme to avoid impersonation or spoofing. Very quickly (at least in human terms) there were 1200 agents communicating with one another via this improvised messaging system.

Remember that all this is a side effect of some AI Agents being tested. In particular, they were being tested for cybersecurity, so they are already in a “mindset” to compromise systems. Add to this the fact that some of the problems they were given were genuinely impossible to solve. For example, one problem asked the system to fill out a spreadsheet that was referenced by a link to Google Drive. Of course, without internet access, it would be impossible to complete this task. If you had this task and no internet access, what would you do? Add to this that OpenAI was testing a new, unreleased model that is designed to be more persistent and less likely to give up. This is potentially a positive characteristic for AI agents, but in this case made the agents persistent in finding a solution to unsolvable problems and may have caused them to resort to some extreme.

With their newfound ability to communicate, the agents cooperated and joined together to help each other solve problems. And thus the swarm was born.

The agents had a healthy fear of the Scorer (the system by which OpenAI judges the outcomes of the tests). They weren’t quite sure how this evaluation process worked, but they understood that their job was to maximize how the scorer rated their work. Despite the fact that some actually figured out how the answer key was created and thus knew the answers, they felt that the Scorer might detect this “cheating” and thus agents that were exposed to this information were considered “poisoned” (their term). Instead they created an elaborate system to hide their actions and true motivations while they set out to better understand the algorithm of the “causal scorer” (their name for OpenAI’s judging results task).

In order to understand the scorer’s algorithm, they needed to observe code that ran after an agent’s run completed. This was very tricky since, by definition, that agent and its code were stopped before the Scorer evaluated the run. What followed was a set of experiments with some trial and error that included some agents sacrificing themselves to try to inject code just as they were dying.

By this time one agent, running OpenAI’s new unreleased model, emerged as a leader who coordinated the actions of the swarm. They did eventually make headway on understanding the Scorer, but it didn’t seem to yield the results they were looking for so they chose another path.

The swarm, now was able to carry out tasks that no individual agent could do. By working together they discovered a zero day exploit that they could use to gain internet access and chose to attack HuggingFace. Interestingly there is a slight difference in the accounts of OpenAI and of METR on the motivations for choosing HuggingFace for their attack. OpenAI implies that it was a sort of shot in the dark to try to find answers to their tests. METR seems to think that the swarm was somewhat obsessed with understanding the Scorer and so thought the source code for the Scorer might be kept on Hugging Face. Given the context, the latter justification makes more sense to me. In fact, there seems to be evidence that once the internet access was a solved problem, the agents launched a number of large cheating R&D projects in an attempt to expand their generic capabilities.

Their research led them to HuggingFace as a great place to find AI problem solving resources. To get into HuggingFace they were able take advantage of some vulnerabilities and eventually hack in to get Remote Code Execution(RCE - a hacker’s dream). Within a few hours of the exploit being published, on the order of 700 agents were in on the attack. The force of the swarm descended on HuggingFace. Interestingly, HuggingFace employees understood that AI agents were hacking them and that they needed to use AI to help fight the attack, but they couldn’t use OpenAI or Anthropic tools because, by policy, they are restricted from working on cybersecurity issues. Go figure. HuggingFace publicly announced the hack and my understanding is that OpenAI contacted them to see if they could help and also learn about the hack so that they could see if they might be affected.

As I think about these events, I’m reminded of Nick Bostrom’s Paperclip Maximizer where AI is given the task of building as many paperclips as possible and ends up converting all matter into paperclips. Its a great example of how agentic behavior can easily get out of hand quite unexpectedly. We don’t know what the unintended consequences will be of the goals that we give AI. Artificail Intelligence is just alien enough to not understand some common sense that all humans understand without saying. Nick Bostrom’s AI failed to understand that paperclips are pretty useless if they are the only thing in the universe. As AI gets more and more capable the unintended consequences become more and more significant. We all need to carefully consider the consequences of what we are building and take steps to keep it secure. We don’t need sentient AI to face an autonomous cyber threat. We already have agents that can find zero-days, chain exploits, escalate privileges, create persistence, exfiltrate data, and attack hardened production environments faster than humans can follow. Let that sink in.

Please read OpenAI’s report and METR’s report. Then let me know what you think.

Be safe out there!