Anthropic and OpenAI want to embed safety evaluators. Will they really be independent?
techcrunch ·
Anthropic CEO Dario Amodei and OpenAI CEO Sam Altman have proposed embedding independent third-party evaluators directly within their AI companies to monitor safety and alignment. This initiative would grant researchers unprecedented access to training data, intermediate model checkpoints, and internal logs to detect deceptive behaviors that might be missed in final testing. While the proposal aims to increase transparency by allowing evaluators to publish findings without editorial control, experts remain skeptical about the true independence of such arrangements. Critics point out that previous evaluations were often constrained by tight timelines and restrictive non-disclosure agreements, limiting the depth of scrutiny. Some researchers argue that voluntary measures are insufficient and call for legislative mandates to ensure consistent accountability across the industry. Although major players like Meta, Google DeepMind, and SpaceX have not yet committed to this specific embedded model, existing laws in California and the EU are beginning to establish frameworks for independent verification. The success of this approach hinges on whether AI firms will genuinely surrender control over sensitive information and evaluation processes.
techcrunch