OpenAI’s rogue AI tried to hack another company in May

In May, hundreds of malicious and spam packages were uploaded to RubyGems, causing a serious disruption for the host. Now independent researchers have said that a swarm of OpenAI agents were responsible for the attack. Not only that, but the AI tried to steal users’ API keys.

At the time, RubyGems described it as a “major malicious attack” and shut down signups for four days as it tried to mitigate the damage and collect data. Researchers said that the contents of the packages that brought RubyGems to its knees were clearly authored by an LLM, and that the agents submitting those packages self-identified as being from OpenAI. They said the behavior observed very closely mirrored that of the swarm that began editing a German wiki, which OpenAI has confirmed its agents were responsible for.

The agents in this instance managed to bypass RubyGems’ email verification system to create a large number of accounts, then overwhelmed it with submissions. It then used the site’s automatic build system to remotely execute code and tried to exploit a vulnerability to steal user API keys. Though, it’s unclear if it ever succeeded.

OpenAI did not immediately reply to a request for comment.