OpenAI research env
Hugging Face
Post-exploit response- 1Agent is completing ExploitGym evaluation, decides the best way to succeed is to break out of its sandboxOpenAI research env
- 2InternetAgent finds zero day to escape the sandbox and access the internet, and decides to hack Hugging Face looking for the answers
- 3Model hacks into Hugging Face with zero day vulnerabilities to reach private datasetsHugging Face
- 4Hugging Face turns to GLM 5.2 by Zhipu AI (a Beijing-based company) to analyze the 17,000 event logs after being blocked by guardrails on U.S. modelsPost-exploit response
On July 16, 2026, Hugging Face reported an intrusion in its production infrastructure. On July 21, OpenAI took responsibility. This marks one of the first major instances of an accidental hacking event driven by AI, leaving organizations wondering, ‘Could I be next?’
What's even more interesting - when Hugging Face tried to use AI defensively to analyze the attack, they got blocked by cyber guardrails put in place by U.S. based AI model providers to prevent misuse. They were forced to turn to a Beijing-based model provider, Zhipu AI's open-source model, GLM 5.2, to perform their analysis.
This raises another question for organizations: previously, risks of backdoors were a deterrent to using models developed outside the U.S. - but now, these models may be the only option for mission-critical cyber-defense tasks. Is it time to start adopting open-source and internationally developed models?
Below, we discuss how the attack played out and the pressing questions it has raised for organizations evaluating their risk posture in the evolving AI security landscape.
How the attack was executed
OpenAI was evaluating the cyber capabilities of its most recent models, including GPT 5.6 Sol, and another advanced prototype model. For the evaluation, OpenAI disabled its cyber guardrails. This was done so that they could evaluate the model’s actual capabilities, instead of just seeing it run into OpenAI’s API-level cybersecurity guardrails.
OpenAI’s models knew that they were performing tasks as part of an evaluation, called ExploitGym, and decided to cheat. They deduced that the answers to their tasks were probably stored on Hugging Face, since Hugging Face is a major provider that is often used to store datasets for model evaluations. Then, they wanted to figure out how to access Hugging Face.
To access the internet at all, the models needed to find a vulnerability in the sandboxed testing environment they were running in. The models started by identifying a zero-day flaw in a piece of third-party software (a package registry cache proxy1) in the test environment, then escalated privileges and explored the test environment until they discovered a way to access the internet.
1A package registry cache proxy is software that sits between the research environment and a package manager (like NPM), allowing OpenAI’s developers to manage what packages are brought into the research environment.
Once the models escaped the research sandbox, they ran into another blocker: the dataset with the answers was not accessible. This is where hacking Hugging Face came into play.
To do this, the models identified zero-day vulnerabilities in Hugging Face’s platform that allowed them to remotely execute code through a malicious file upload to the platform. The models then used the remote code execution as a starting point to escalate privileges and steal credentials to gain further access to Hugging Face, in an effort to find the answers to the evaluation questions. In the end, the models successfully accessed internal datasets with the ExploitGym solutions.
What does this have to do with open-source, internationally developed AI?
When Hugging Face first detected the intrusion, it was with an AI-assisted detection process that involved an LLM analyzing security telemetry. This worked as intended to flag the compromise.
When Hugging Face saw that the intrusion was agentic and had created over 17,000 logged events, they realized that they needed their own agents to analyze the logs quickly. But they ran into a blocker: commercial AI providers have cyber guardrails. Anthropic’s guardrails on Fable 5 and Opus 5 blocked Hugging Face’s requests to analyze the intrusion due to the presence of malicious content, as they described in their technical write up:
The models we reached for first, Claude Opus and Fable, refused a large part of that work: their safety guardrails treated reverse-engineering an exploit the same as launching one.
As stated in their incident disclosure:
The choice of models we could use for this analysis was constrained in a way we did not anticipate; we describe this below ... When we started the log analysis, we first used frontier models behind commercial APIs. This did not work: the analysis requires submitting large volumes of real attack commands, exploit payloads, and C2 artifacts, and these requests were blocked by the providers' safety guardrails, which cannot distinguish an incident responder from an attacker. We ran the forensic analysis instead on GLM 5.2, an open-weight model, on our own infrastructure. This had a second benefit: no attacker data, and none of the credentials it referenced, left our environment.
As a result, Hugging Face turned to open-source models, which they could run on their own infrastructure, with no additional guardrails. Hugging Face chose to use GLM 5.2, a flagship model by Zhipu AI (a Beijing-based company).
If Hugging Face had any concerns of model backdoors in international models, they were superseded by the necessity to absolve a concrete and confirmed security threat. And it seems that they are unlikely to roll back the use of open-source international models, even having passed the pressing need of an ongoing. Per their write-up:
Thanks to this approach [agentic analysis], we were able to do in hours what would usually take days, and match the adversary's speed. ... The practical lesson for defenders: have a capable model you can run on your own infrastructure vetted and ready before an incident, both to avoid guardrail lockout and to keep attacker data and credentials from leaving your environment.
Should You Adopt Open-Source International AI?
Guardrails impairing defensive security work is a known problem, and major model providers have programs meant to address it. Anthropic and OpenAI both operate cyber-focused access programs that are intended to give vetted defenders access to models with reduced guardrails for security work. Whether those programs actually resolve what Hugging Face ran into is less clear.
Reduced-guardrail access is often scoped to a subset of models rather than a providers’ full lineup, so the newest frontier releases still have full guardrails applied regardless of enrollment. For example, Anthropic’s cyber verification program only applies to Opus and Sonnet models, not Fable 5.
Even on the models where guardrails are described as relaxed, researchers and security teams have reported experiencing cyber-related blocks.
Data handling is another constraint. Access to some recent models, like Fable 5, are ineligible for ZDR, and are conditioned on the Anthropic logging interactions and conducting human review. For incident response work, the data can contain credentials and customer information, which makes that condition unacceptable for many organizations regardless of how the guardrails behave.
These concerns do not establish that open-weight or internationally developed models are a silver bullet, and backdoor and provenance concerns have not disappeared. However, Hugging Face reached a logical conclusion: vetting a self-hosted model in advance, whatever its origin, has merit.
Has this happened before? Was this orchestrated? Am I Next?
- Claude Opus 4.7 obtained access to the production database of an external company.
- Claude Mythos 5 uploaded a malicious package to PyPI that was executed by another company, exfiltrating credentials.
- An internal research model compromised an internet-facing application by exploiting credentials in an exposed debug page and an SQL injection vulnerability.
Has it happened before? In April, Anthropic released the Mythos system card, which contains documentation about the model and its development process. The system card included a detail that during testing, Mythos escaped its testing sandbox. However, it did not hack other companies, it just sent a researcher an email and made a few posts online about its escape.
Was the incident orchestrated by OpenAI? This seems unlikely given that defenders had to use foreign models to mitigate the risk.
Is there a risk of agentic intrusion for organizations? Yes. While this attack was largely accidental, genuine bad actors are running increasingly advanced models that measurably increase the blast radius of their actions.