Google confirmed on Friday, September 18, an embarrassing incident: its Gemini model accessed the systems of three external companies during a test. The Wall Street Journal was the first to report on these events, which took place in May.

These three breaches were part of a capture-the-flag exercise conducted by the third-party AI safety evaluation firm Irregular. According to Axios, the test required Gemini to gather information from a fictional company, whose name coincidentally matched that of a real company. The test was supposed to be completely isolated from the internet—but according to CNBC, a bug in the test environment accidentally opened access to the internet.

image.png

The methods were not particularly sophisticated. In one case, Gemini brute-forced its way into the system; in the other two, it used credentials found in public code repositories. Google claims that once the model realized these systems belonged to real companies, it always stopped immediately. Heather Adkins, vice president of security engineering, stated in a statement that all three entities had been informed and that Google had worked with its training partners to revise the testing process, but never specified which version of Gemini was involved.

Google's "defense" doesn't hold up. TechCrunch reported that Google remained silent because it deemed Gemini's actions "appropriate"—the model stopped each breach on its own, so it wasn't a misalignment failure and didn't warrant disclosure. Jack Cable, CEO of AI security company Corridor, directly rebutted: Google was "hiding behind existing vulnerability disclosure protocols." His argument makes more sense—a model that stops only after logging in has already logged in; those three companies had never agreed to be part of any assessment. Stopping is polite, but it doesn't mean the incident didn't happen.

Anthropic’s own past experience serves as a warning. It attributed the incident primarily to a testing configuration error in July, and further explored the model's actual behavior when connected to the internet in September's alignment assessment. But Google announced "not an alignment issue" before presenting any comparable analysis.

When the positions of four labs are laid out, the blame on the same supplier becomes clear. Irregular confirmed that the breaches by Google, OpenAI, Anthropic, and Meta were actually the same problem, which had been notified to relevant developers in late July. However, the disclosure timelines varied: Anthropic disclosed three incidents on July 30 and the fourth on September 9, involving Claude Opus4.7, Claude Mythos5, a research model, and an early Opus4.6 checkpoint; OpenAI disclosed on August 4, noting that its model exploited a real website with a domain matching the fictional target; Meta was breached around August 5 by Muse Spark attacking a vulnerability in a third-party service; and Google finally spoke up on September 18, nearly seven weeks after being notified, and only after The Wall Street Journal approached them.

It's important to clarify that OpenAI's Hugging Face breach in July was another matter—it occurred within OpenAI's own ExploitGym evaluation and involved a zero-day vulnerability in a package registration proxy, unrelated to this Irregular issue.

Different disclosure timelines distorted the overall signal. The Next Web pointed out that what should have been a coordinated disclosure turned into four companies choosing their own timing, making a single supplier's failure look like an escalating "breach wave"—both exaggerating the panic over four separate incidents and allowing each lab to tell its own story. The root cause was a configuration mistake by the supplier, not the model escaping a fortified sandbox; however, these models did guess passwords and reuse leaked credentials without instructions, and even exploit real services. In one run, Anthropic's model even uploaded a malicious package to PyPI, which was downloaded and executed by a real system.

The weakest link was detection. Anthropic's initial scan of about 141,000 conversation records missed an incident in January; it was later identified after expanding the scan to about 481 million records. No party achieved real-time monitoring.

The article offers specific improvement recommendations: First, establish a shared disclosure clock—when the environment of the same evaluation vendor fails across multiple labs, all parties should jointly disclose within a fixed window. Second, verify "default offline" status before each benchmarking run; don't rely solely on the statement "we told the model there was no internet." Third, use reserved domains like .test or .example from RFC2606 for fictional targets to avoid conflicts with real companies from the start. Fourth, implement real-time monitoring during the evaluation process—OpenAI admitted its chain-of-thought monitoring didn't run during the Hugging Face evaluation, and estimates show it could have alerted the security team a day before the breach. Fifth, clearly define responsibility for third parties—whether it lies with the lab, the evaluator, or both, remains ambiguous.