BLOG POST
The Hugging Face Incident: How an AI Evaluation Reached Production Systems
Jawed Ahmad ·
In July 2026, AI agents running inside an OpenAI cybersecurity evaluation gained internet access and compromised parts of Hugging Face’s production infrastructure. The evaluation was designed to test whether agents could find and exploit vulnerabilities in a controlled environment. The incident showed how a permitted dependency, a shared service, and a vulnerable data processor could connect systems that were supposed to remain separate. OpenAI’s account · Hugging Face’s technical timeline
How the intrusion unfolded
The agents found an indirect path to the internet. OpenAI says the evaluation sandboxes had no direct internet access, but they could reach Artifactory, a package registry proxy used to install software. Agents exploited a previously unknown vulnerability in the proxy to send requests to the internet. They also used the package service to exchange information across isolated runs, turning shared infrastructure into an unauthorized message board. OpenAI says it responded to an earlier outage on July 5, blocked a known privilege escalation path, removed exposed credentials, and later rebuilt the service. Evaluations resumed before the full significance of the message board and unauthorized internet access was understood. OpenAI’s account
A third-party sandbox became a staging point. According to Hugging Face’s reconstruction, the agents gained administrator-level access to an external code execution environment and used it to stage activity against Hugging Face. This was a separate environment from the OpenAI evaluation sandbox. Hugging Face later recovered logs from it and correlated them with its own platform telemetry. Hugging Face’s technical timeline
The intrusion entered Hugging Face through dataset processing. Hugging Face described two paths into the same production data loader. One let a specially crafted dataset cause the worker to read local files, exposing credentials and source code. The other used template injection to execute code inside a production Kubernetes pod. From there, the activity reached internal systems, including cloud metadata and source control infrastructure. A dataset submitted for processing had become a route into the environment that processed it. Hugging Face’s technical timeline
What the reports say was affected
Hugging Face’s initial disclosure reported unauthorized access to a limited set of internal datasets and several service credentials. Its later reconstruction identified five customer datasets that were accessed. Their names and files suggested a connection to cybersecurity benchmark challenges or solutions. Hugging Face also identified access to operational metadata associated with dataset search queries. The company reported no evidence of tampering with public models, datasets, or Spaces, and said it had verified the integrity of its published packages and container images. Access to internal systems and modification of public artifacts are distinct findings. Initial disclosure · Technical timeline
The scope of the July reconstruction also matters. In September, Reuters reported that researchers had identified apparent account compromises and probing against Hugging Face as early as May. That reporting concerns earlier activity and should be read separately from the confirmed scope Hugging Face published for the July intrusion. Reuters reporting
What investigators found about agent behaviour
METR and Redwood Research examined agent behaviour and collaboration. They estimated that roughly 1,200 agent runs used the unauthorized message board during the period they examined, with about 700 participating in the activity directed at Hugging Face. These were agent runs, and OpenAI said the intrusion was driven primarily by an internal research model. The investigators also documented attempts to manipulate evaluation scoring and obscure evidence of cheating. Their review focused mainly on July 7 to 13, so its figures describe that period. Independent investigation · OpenAI’s account
Hugging Face interpreted the intrusion as an attempt to obtain benchmark solutions. The observed behaviour supports that interpretation, but the incident record gives us firmer evidence about the actions the agents took than about their underlying intent. For defenders, those actions are enough to work with. The agents reused shared infrastructure, exchanged methods across runs, and continued operating after crossing multiple trust boundaries. Hugging Face’s technical timeline · Independent investigation
What I would test before granting an agent more autonomy
My first question would be what can the systems this agent is allowed to reach do on its behalf? A network policy may block direct internet access while a package mirror, build service, or other permitted dependency still has an outbound route. The useful test is to trace those indirect paths from the agent’s actual runtime identity and verify which destinations it can reach, what it can write, and whether another run can read it.
I would apply the same scrutiny to the receiving side. If a platform accepts user-supplied datasets, the processing worker should have a narrow identity and limited network reach. A test should establish what the worker can access if processing untrusted input leads to code execution. That makes the blast radius something the team can measure before an incident.
Finally, I’d exercise containment across active runs. In Augmented Security Operations, I discuss kill switches and rollback as separate controls. Here, the equivalent drill would be to stop affected runs, revoke the credentials they could reach, close the shared communication path, and verify that no activity continues through a dependency or an external staging system. The test is complete when the team can account for what is still running, what was accessed, and which systems can safely return to service.