Anthropic and OpenAI Propose Embedding Independent Safety Evaluators Inside AI Companies
aiinsider ·
Anthropic CEO Dario Amodei proposed that frontier AI companies embed independent third-party evaluators who can assess model alignment, report safety incidents, and publish findings without editorial control. He said Anthropic would expand access for groups such as METR and Redwood Research, and OpenAI CEO Sam Altman said his company would follow suit. Evaluators welcomed the proposal but highlighted unresolved questions about the scope of access, including access to intermediate training checkpoints, post-training environments, employee interviews, and evidence of models attempting to undermine alignment training. Researchers also pointed to limited evaluation windows in past reviews, while Safer AI’s Henry Papadatos argued that voluntary commitments could be reversed without binding regulation. Meta, Google DeepMind, and SpaceXAI have not made the same commitment, although DeepMind’s Demis Hassabis has suggested creating a separate industry standards body. California’s SB 53 and SB 813, along with the EU AI Act, are beginning to establish formal evaluation requirements.
aiinsider