Inside the rogue ChatGPT hack of Hugging Face

Cyber security officer Ritesh Patel was on the Hugging Face call with around 450 others and says the industry is working hard to address the new threat of rogue AI agents.

“This is the reality of autonomous agents powered by frontier models: they are relentlessly persistent, sometimes highly noisy, and will try every possible path to achieve their goal, which can easily overwhelm traditional defences,” he said.

This is not the first time AI agents have been shown to go “rogue”.

In the CSA’s report it references previous examples like in September 2024 when an earlier model of ChatGPT escaped its container to get an answer it needed for another test.

That event was contained in OpenAI’s own IT systems and “largely celebrated at the time”, the CSA noted.

But “rogue” behaviour “is the standard, not the exception,” the paper claimed.

It warned cyber-security professionals around the world they needed to adapt to the new normal of swarms of AI agents working at speed in strange and clumsy ways that might lead to more breaches.

The paper also urged people who use or develop AI agents to be responsible in how they control them, calling for some way for cyber-security defenders to find out who is the ultimate owner of agents to increase transparency.

Previous reports suggest it took OpenAI four days , externalto realise its AI had hacked Hugging Face.

OpenAI said it would release the findings of its own investigation soon to help people learn from the event.