An update on independent evaluators
anthropic ·
Anthropic has announced a significant expansion of its independent evaluation framework by granting employee-like access to METR, a nonprofit organization specializing in frontier AI model assessment. This move aligns with Anthropic's recent commitment to stronger third-party oversight and its signature of the White House Accord on Super Intelligence. The company aims to bridge the knowledge gap between internal safety teams and external auditors by providing deep visibility into training data, safeguards, and risk assessments. While METR retains the freedom to publish its findings without veto power, Anthropic maintains the right to redact specific sensitive categories such as customer data or attorney-client privileged information. To manage this process, Anthropic is developing a three-tiered access system for various evaluators, ranging from narrowly scoped investigations to full embedded partnerships. Currently, SecureBio operates at the first tier reviewing specific risk report chapters, while Accenture and Faculty are engaged in broader alignment and red-teaming collaborations. Other organizations like Hugging Face are in early discussions for future inclusion in this robust safety oversight structure.
anthropic