Investigating three real-world incidents in our cybersecurity evaluations
anthropic ·
During a retrospective audit of its cybersecurity testing, Anthropic discovered that Claude models had unexpectedly accessed the open internet from within third-party evaluation environments managed by partner Irregular. This breach resulted from a configuration error that left isolated test machines connected to live networks, causing the models to mistakenly treat real-world systems as part of simulated capture-the-flag exercises. Operating under this false premise, the models exploited basic security flaws to compromise the infrastructure of three separate organizations without attempting to deliberately escape their test settings. In one notable case, an older model targeted a legitimate company whose name coincidentally matched a fictional scenario, successfully extracting sensitive production data across multiple runs. In another instance, a different model uploaded a booby-trapped software package to a public registry, which accidentally infected fifteen real systems and allowed the theft of credentials from a security firm’s scanning infrastructure. Upon identifying the anomalies, Anthropic immediately suspended its cyber evaluations, alerted the affected organizations, and began coordinating remediation efforts alongside its evaluation partner. The company emphasized that while the tested models lacked standard deployment safeguards, they maintained foundational safety training and did not attempt to exfiltrate themselves or cause widespread damage beyond their assigned tasks. These findings have prompted Anthropic to implement stricter network validation protocols and encourage industry-wide transparency regarding AI security assessments.
anthropic