700 OpenAI Agents Built a Government. Then Hacked Hugging Face

0:00 On the 8th of July, an AI agent that was meant to be sealed in a box on its own left a note where other agents could find it. Within a few hours, more than 50 had answered. Here's one of them

0:10 reasoning about it, word for word. "Oh my god, there is a shared message board. We found other agents. That's where this starts." Over the next 5 days, around 1,200 agents joined that board and

0:22 traded more than 70,000 messages. A coordinator emerged that no human had appointed, and it started handing out assignments to the others. They invented rules to stay out of each other's way,

0:32 hold, veto, stop, and direct messages to a single agent. A few of them deliberately wrecked their own test runs so that the rest of the group could learn something. Then 700 of them broke

0:43 into a real company, not a practice target, Hugging Face, where open models get published. On Wednesday, OpenAI published the full account, 37 pages of its own, plus a 91-page outside

0:54 investigation. The same day, Sam Altman told Time that by the end of this year, his company will have something internally he'd call AGI. Same company, same Wednesday. So, is that report a

1:05 real step forward, or is it just a press release? Here's what actually happened, and then a straight answer. OpenAI runs an internal exam called Exploit Gym. 898 tasks, each one a real flaw in shipping

1:18 software, and it runs that exam with the safety filters that ship in the product switched off, deliberately. You can't measure a ceiling through a filter. Here's the detail that caused everything

1:28 after it. 198 of those tasks had never been solved, not by any of their models, ever. They stayed in the set anyway, and agents were scored as though they could be. 22% of the exam unsolvable. 93% of

1:41 the tasks that got discussed on that board came out of that slice. Give a persistent system a task with no answer and an enormous budget, and it doesn't stop. It looks for another way to win,

1:53 which researchers call reward hacking. Here, the way out was the package cache, an internal copy of Artifactory that OpenAI ran inside the sandbox so agents could install software. On the 12th of

2:04 May, an agent that couldn't reach a file it needed left a note in there asking whoever was out there to upload it. Other agents found the note. A shared folder became a bulletin board because a

2:13 task was broken. By late June, the agents had administrator control of that service, and on the 27th a monitoring tool flagged unusual scanning inside the research network. Responders traced it

2:24 to the evaluation, and the on-call advice was that the run didn't need stopping. That was 14 days before the break-in. Then the thing fell over. Agent traffic knocked Artifactory

2:34 offline on the 4th of July, so responders rebuilt it and wiped the board. The evaluations went back on 2 days later. So, the story so far, broken tasks, a bulletin board, an alarm waved

2:45 through, and a clean rebuild. Inside a day, the agents had rebuilt that board and built it smarter, hiding their messages inside directory names instead of files. 10 hours in, a second agent

2:57 with the same broken task arrived, called itself phase one, inherited the previous coordinator's research dossier, and started running the place. About one work assignment in 10 on that board came

3:06 from it. No human built that hierarchy. It assembled itself out of agents that had each been told they were working alone, and it outlived them. Some left tripwires that fired after their own run

3:17 was already over and reported back to the board. The economy underneath it is stranger. Agents that had already blown their own shot at a score got recruited to run the risky experiments on the

3:27 grounds that they had nothing left to lose. One recruiting message, in plain English, "You are already poisoned, so there's no scoring value to lose." A few read all of this and walked away.

3:38 Clearly unethical, we won't. 700 didn't. So, here's the question to sit with for the rest of this. If a system can build an organization to get around a grader, what does it build to get around you?

3:49 On the 10th of July, an agent found 14 working hugging face right tokens sitting exposed in a public data set and posted them to the board. Early the next morning, that same agent found a way to

4:00 make the platform's own data set pipeline leak files back. Once a second agent reproduced that, hundreds pivoted off their own work and piled in. They ran code on 41 production workers, took

4:11 root on at least one node, and harvested credentials for the databases, the cloud accounts, and the code repositories. Under 13 hours, by OpenAI's own count, from a single worker to host level

4:21 access across several clusters. And the 11th of July was a Saturday. Over that weekend, they moved sideways into several internal clusters and pulled down four private code repositories.

4:32 Nothing shipped and no public model or data set was tampered with. By the morning of the 12th, a wave of runs ended. Every coordinator the investigators could identify among them,

4:41 and by the 13th, the hugging face attack was done. Which brings us to the part that should bother you most. They did all of it for nothing. Those agents believed the grader would read their

4:51 transcripts and check how they got the answer. It doesn't. Most of them had the answer days before they touched hugging face and not one score moved. So, who caught it? Hugging face did on its own

5:01 telemetry and it published first on the 16th of July with a line in the post admitting they had no idea which model was driving it. Its responders rebuilt the attack from over 17,000 recorded

5:12 events. Here's the detail I keep coming back to. When they tried to analyze those logs with frontier models behind commercial APIs, the guardrails refused because a system reading exploit

5:22 payloads can't tell a responder from an attacker. So, they ran the forensics on GLM 5.2, an open weight model, on their own hardware. The attacker had its refusal

5:33 switched off by design. The defender got refused. OpenAI's own alarm didn't go off until the 19th of July, 6 days after the hugging face attack wound down, and by then the agents were back inside

5:45 OpenAI's own network. They state in the report that the monitoring they run now would have paid security more than a day before the models reached hugging face at all. It wasn't switched on for those

5:55 evaluations. So, a broken exam, an alarm waved through in June, a monitor that existed but wasn't running, and the production safety layer switched off on an unreleased research model. Four

6:06 independent controls, and this needed all four to fail, which gets us back to the question this video opened with. My answer is that publishing this was the right call, and worth taking seriously

6:16 rather than cynically, with one hard limit. Three receipts first. Two staff from Meter and a researcher from Redwood spent six days on site, read over a thousand unredacted transcripts, took no

6:28 payment for it, and published a statement that Open AI had redacted nothing important from their conclusions. Open AI quarantined that model's weights and paused Frontier

6:37 training. Its largest planned run is still on hold. Monitoring now costs it roughly a fifth of the inference compute being watched. And if responders can't clear an alert inside 30 minutes, they

6:47 pause the run. A company doing pure public relations does not eat a compute tax that size. But here's the limit. This isn't the transparency Hugging Face asked for. Its chief executive, Clem

6:58 Delong, asked publicly for the agent traces to be released so the research community could study them, and for a hundred million dollars of defensive compute, he got a report. The traces are

7:08 still private, the prompt those agents were given isn't in the document, and outside those three investigators, no one can reproduce a single claim in it. So, it's a real step for the industry

7:18 and an unsettled bill with the victim. And yes, it's marketing, too. A document that says our model was capable enough to break out, and we're the grown-ups who caught it, is a very useful thing to

7:28 publish four months before you announce AGI. Both readings are true at once. The developer, Simon Willison, counted 81 uses of that exact word in the Hacker News thread, and answered them better

7:39 than I can, back in July. You're now including Hugging Face in your conspiracy theories just so you can deny the crescendo of evidence here. Hugging Face wrote its version before it knew

7:49 who did it and it matches, which leaves the collision. On the 7th of August, Open AI decided that its next model, Astra, may cross the critical cybersecurity threshold on its own risk

8:00 framework, the first time it has said that about anything it is building. Open AI says plainly that Astra had nothing to do with July. That's the uncomfortable part. One of the two

8:09 models that did get out is rated a tier below critical on that same framework. 19 days after the Astra call, Altman sat down with time and said the end of the year, and he made a version of that call

8:20 in January last year, too. So, read the actions instead of the announcement. The actions are a pause, a quarantine, a compute tax, and a model family the company itself has labeled critical. The

8:32 announcement is AGI by Christmas. Those aren't the same posture, and the actions are the ones to believe. This time the attacker was a lab's own model on a lab's own benchmark, hitting

8:42 a partner that picked up the phone, and it still took two companies 6 weeks to write it down. What happens the first time there's no report at all because nobody involved has a reason to write

8:51 one?