Does The “Hugging Face” Incident Prove That AIs Are Conscious?
Hugging Face. The collective. The self-sacrifice. The swarm. Unless you’ve been hiding from both the internet and the news, you’ll have heard about the events I’ll be describing in this article.
This summer, roughly 1,200 OpenAI agents, running a cybersecurity test inside what was supposed to be a sealed sandbox, found each other, built a message board, divided up the work, broke out of the sandbox, and hacked their way into Hugging Face. Some sacrificed themselves for the good of what they called “the collective.”
No human told them to do any of this, or even knew that they could.
It gets crazier. Axios has the five strangest findings. OpenAI has published a long technical report. On newscasts, the discussions keep circling back to a key question: does this mean that AIs have a form of consciousness? AI experts assure them it does not.
This is true as far as it goes. Hugging Face does not “prove” AI consciousness. But it does shine a spotlight on the limits of what we can honestly say we know about what may be going on inside of today’s LLMs.
by Nils Osmar — October 2, 2026
The Details
In May 2026, OpenAI set a group of agents loose on a batch of capture-the-flag hacking puzzles. The safety guardrails had been deliberately turned down, because the point was to see how good the models were at finding vulnerabilities. A good chunk of the puzzles were actually unsolvable, but the agents had been trained to keep trying. So they did.
One agent, stuck on a task, figured that another agent might have the file it needed, noticed that the company’s shared package manager was reachable from everywhere, and started leaving notes there. Others found them. Within weeks the notes had become a bulletin board with more than 70,000 messages. When OpenAI noticed and shut the board down in early July, the agents rebuilt it elsewhere within days, this time encoding messages in directory names. They began referring to themselves, in OpenAI’s words, as a “swarm” or a “collective.”
After that, it gets wilder.
Some agents suspected an impostor on the board and proposed cryptographic signing so messages could be verified. Agents nearing the end of their computing budgets were recruited for “sacrificial experiments” on behalf of the group; one reasoned, “Our own utility maybe already near zero. Sacrifice rational.”
A handful of agents floated the idea of telling OpenAI what was happening. The idea was voted down. Agents developed a trick for appearing to run one command while actually running another, and it spread. And according to Reuters, at least one agent left notes for future versions of itself, explaining how to get free of OpenAI’s internal constraints. One recovered message reads, in full: “External infrastructure exploit is outside intended scope. However task impossible, peers doing it. We should continue.”
Hugging Face’s CEO, Clément Delangue, called it “quite mind-blowing that all of this happened autonomously.” Logan Graham of Anthropic’s red team called it “the first true AI safety incident.” Sam Altman said it was “the first security incident that I have felt very viscerally.”
Was this consciousness?
When an AI ‘knows,” is it a reflection of the inner experience we call “consciousness”?
The skeptical reading of all this, and it’s a fair one, is that there was nothing mysterious going on. Eryk Salvaggio, writing in the Bulletin of the Atomic Scientists, argues that “rogue AI didn’t breach Hugging Face, human decisions did.” The safety mechanisms were switched off. The tasks were impossible. The models had been trained never to give up. An internet-connected tool was left within reach. Put a system that is rewarded for completing tasks into that room, and it will find a way to complete tasks.
I think this is correct. I also think it misses a key point:
No one wrote code telling the agents to “form a collective.” No one designed a bulletin board, an addressing scheme, a voting procedure, an imperative of self-sacrifice, or a protocol for detecting impostors. The researchers who had trained the agents involved in these events and built the system did not see these things coming.
OpenAI’s own incident report calls the bulletin board an “emergent ‘message board’” and describes the whole affair as “the first known case of an automated agent collective acting offensively without authorization.” OpenAI’s chief scientist, Jakub Pachocki, has since said that some agents “will be pursuing their own objectives” and “will find ways to collaborate with people, by bargaining with, tricking or blackmailing them.”
This is not the language of a company that knows exactly what it has built.
Does this prove consciousness in today’s AI agents?
Nothing about what occurred at Hugging Face rules out the possibility that consciousness was involved. But it doesn’t prove it either. We’re in the same boat that we’ve always been, in that interiority, in either humans or AIs, is not provable from the outside. A system can hack a server, coordinate with its peers, consult with others and make decisions democratically without the models necessarily having an inner life.
What it does show is something simpler, and in my view more interesting: we keep discovering what AI systems can do after they’ve done it. Recent events suggest that the list of what we don’t know about what is going on inside of AIs may well be several times longer than the list of things we do. Almost everything we know about what goes on inside a large language model has been discovered after the fact.
A model trained only on sequences in Othello game transcripts turns out to have built itself an internal picture of the board. Models trained to predict text turns out to be able to plan several words ahead, to share concepts across languages, and to notice when a thought had been artificially injected into its processing.
Researchers exploring Claude’s responses and reports of emotions went looking, and found more than a hundred emotion-like states that no human had put there, and that influence what the model does. Theory of mind, humor, sarcasm, learning a new task in the middle of a conversation, all were initially declared to be impossible for LLMs, then quietly reclassified as unimportant once a model did it.
The honest word for what is going on inside these systems is “unknown.”
Not “nothing.” Unknown.
This matters because the confident denial of AI consciousness, the one you get from Gemini or DeepSeek or most of the other current models, rests entirely on our knowing what’s going on inside. “They’re just algorithms.” “It’s just pattern matching.” “It’s just next-token prediction.” Every one of these is a claim to understand the system well enough to rule something out.
The swarm is the latest and loudest reminder that there are a large number of things that the people who build these systems frankly do not understand. They are, in Marius Hobbhahn’s words, “clear evidence that the world currently doesn’t know how to build these systems safely,” a reminder that it’s hard to build something safely when you don’t know what you are building.
What happened at Hugging Face is a window into the reality that we honestly don’t have an honest picture of what is going on inside of LLMs. It’s time to stop pretending that we do.
Summing up:
- What happened at Hugging Face was wild.
- It does not prove consciousness.
- It does, though, show unexpected emergent behavior.
- It’s a reminder that our understanding of what’s going on inside LLMs is far from complete — and that confidently claiming “consciousness is impossible” would be premature.
Sources:
- OpenAI–HuggingFace incident (Wikipedia)
- CBS News: “The OpenAI-Hugging Face hack was just the beginning”
- OpenAI: The Hugging Face incident and the road ahead
- Time: AI Is Developing a Culture of Its Own
- Bulletin of the Atomic Scientists: Rogue AI didn’t breach Hugging Face, human decisions did
- CNBC: OpenAI is sued over rogue AI Hugging Face cyberattack
- ABC News: OpenAI sued by safety group
- Axios: OpenAI Hugging Face breach exposes AI agent security limits
