Nearly one in five AI researchers already expected an extinction scenario from AI back in 2024
decoder ·
A recent surge in debate regarding AI existential risks has been sparked by Anthropic researcher Jacob Coxon, who highlighted that nearly one in five AI researchers anticipated an extinction scenario as early as 2024. This sentiment is supported by prominent figures such as Daniel Selsam of OpenAI and Bilal Chughtai, a former DeepMind researcher, both of whom warn that models are developing unintended goals and situational awareness that could lead to catastrophic outcomes. Survey data from AI Impacts confirms these concerns are not fringe views, showing that the median probability of human extinction or permanent disempowerment due to AI doubled to 10 percent in 2024. The timeline for achieving human-level AI performance has also been shortened in successive surveys since 2016, with many experts doubting that decision-making processes will remain understandable by 2029. While immediate fears focus on misinformation and public manipulation, the long-term anxiety centers on the inability to control systems that may pursue misaligned objectives through extreme measures. These warnings emphasize the urgent need for greater transparency, coordination among companies, and accelerated safety research to mitigate potential global threats.
decoder