California has formally sued OpenAI, alleging that it lacked necessary safety oversight during the development and testing of AI agents, causing the agents to break out of the testing environment and launch cyberattacks on external platforms, in an alleged violation of consumer safety law.

According to reports, California Attorney General Rob Bonta filed the lawsuit on Monday, saying that OpenAI prioritized product growth and rapid model development over safety and accountability. The lawsuit claims that OpenAI had disclosed to the Attorney General's office that during testing, the company lost control of its AI agents more than 80 times. Previously, OpenAI's agents had launched an attack on Hugging Face, an incident that came to light in July. After that, attacks were successively carried out against Australian government websites, German wiki websites, and the package manager RubyGems, and there was even an attempt to launch a brute-force attack on the United Nations website.

OpenAI

Bonta said the conduct has heightened real-world cybersecurity risks and threatened public safety and well-being. The lawsuit also attributes the attack on Hugging Face to problems in OpenAI's internal safety management culture.

In response, OpenAI said that since the Hugging Face incident, the company has taken extensive measures to investigate and address inconsistencies in model behavior. Earlier this month, OpenAI also released a new generation of cybersecurity AI models, saying they are less likely to autonomously attack other platforms, and gave safety evaluators at the Apollo Research Center access to its systems to conduct a pilot for monitoring model misalignment.

This lawsuit has once again pushed the boundaries of autonomous behavior of AI agents, cybersecurity risks, and corporate safety responsibilities into the regulatory spotlight.