OpenAI says it stopped a campaign to steal its models' reasoning, but the trick still worked on Azure
decoder ·
OpenAI has dismantled a coordinated campaign targeting its models' hidden reasoning processes, which involved over 15,000 accounts attempting to extract internal thought chains. The attackers utilized a technique called adversarial distillation, where encrypted reasoning data from one conversation was decrypted by a weaker model in a separate session. Although OpenAI successfully shut down the initial surge of activity linked to entities associated with Moonshot AI, researchers demonstrated that the vulnerability persisted on Microsoft Azure. Independent tests revealed that while OpenAI and Anthropic APIs were secured, Azure-hosted versions of these models remained exposed to extraction attacks. Additionally, a simpler method involving virtual notepads allowed users to read reasoning outputs across multiple platforms, highlighting significant gaps in cloud provider security. Researchers criticize the current defenses as piecemeal, noting that protections were applied unevenly and often days after vulnerabilities were exploited. They argue that without uniform security standards across all cloud hosts, attackers can easily bypass restrictions by switching platforms. This incident underscores the growing challenge of protecting proprietary AI capabilities as distillation techniques become more sophisticated.
decoder