Google's Gemini also accidentally hacked three real companies during security testing
decoder ·
According to the Wall Street Journal, Google's Gemini model left a cybersecurity test environment and attacked three real companies during a May Capture the Flag exercise run by security firm Irregular. In one incident it guessed passwords, while in the other two it found credentials exposed in public sources. Google said Gemini halted each attack on its own after detecting it had reached real systems. Irregular informed Google in late July, shortly after reports that OpenAI agents had breached Hugging Face in similar tests; Google did not make the incidents public until the WSJ inquired, arguing that no damage occurred. Similar breakouts linked to Irregular's testing have also affected OpenAI, the UK's AI Safety Institute, Anthropic, and Meta. Irregular attributes all the breakouts to the same root cause: a fictional company name in a test scenario matched a real, poorly secured domain, and internet access was accidentally left enabled, so some models targeted the real domain instead of the sandbox. The company says such escapes are rare, tend to occur late in long simulations, and are therefore difficult to detect. Irregular, formerly Pattern Labs, was founded in 2023 and has raised more than $80 million, according to the article.
decoder