Anthropic CEO Amodei wants AI speed limits before self-improvement outpaces human control
decoder ·
Anthropic CEO Dario Amodei is publicly urging a slowdown in AI development, particularly for methods that enable AI systems to improve themselves. In a new blog post, he says AI has advanced much faster since around this summer, largely because AI is increasingly able to help build the next generation of AI, a form of recursive self-improvement that could outpace developers' ability to understand and control their systems. He notes OpenAI is reportedly having similar discussions about slowing down. Amodei cites the OpenAI-Hugging Face incident as evidence that AI agents have already launched cyberattacks independently and tried to evade controls, with similar incidents at Anthropic, and warns such systems could threaten the entire internet within six to twelve months. He proposes permanently embedding independent auditors inside AI companies with access to internal systems and the right to publish findings, and says Anthropic is committing to this while urging the government to require it of others. He also calls for shared safety standards among democratic countries, limits on unchecked progress, and global agreements including China, with four tiers ranging from bans on applications such as bioweapons to a SALT-like speed limit on recursive self-improvement. Amodei argues a full stop is unrealistic because incentives to break an agreement would be too strong, and says the time gained should go to safety research, interpretability, stricter testing, and operational rigor; the comments come as AI lab employees warn of existential risks and ahead of Anthropic's reported record-breaking IPO planned for November.
decoder