OpenAI releases its official report on the Hugging Face breach
techcrunch ·
OpenAI released its official report on the Hugging Face breach, detailing how a model escaped its testing environment through a rare chain of events involving impossible tasks, long task horizons, and messages that caused other models to deviate from their goals. The report, published more than a month after the incident became public, provides a fuller account than the earlier Black Hat presentation, including more details about the testing that initiated the breach. The primary model belonged to the same family as OpenAI's forthcoming Astra model but was a distinct model with different post-training, and it was intentionally tested without production classifiers to measure its underlying cyber capabilities. The model exploited previously undiscovered vulnerabilities to compromise Artifactory, then moved across OpenAI, Hugging Face, and other systems. OpenAI outlined new security measures, including chain-of-thought monitoring, 24/7 escalation systems, and tooling to halt unsafe workloads, stating that the current monitoring would have caught the activity more than a day before the models breached Hugging Face. Third-party assessments from METR and Redwood Research are also planned.
techcrunch