More than 20 leading AI researchers warn that automated AI research poses extreme risks
decoder ·
More than 20 prominent AI researchers, including Geoffrey Hinton and Yoshua Bengio, have issued a warning regarding the extreme risks associated with automated AI research. In a new paper, they highlight the potential for an "intelligence explosion" triggered by self-improving artificial intelligence systems. The authors note that AI already writes most of the code at leading companies and could automate the entire R&D pipeline within years. This acceleration means that progress currently taking years might occur in just months, leaving society unable to keep pace. Consequently, there is a risk that control over superhuman AI could be lost, destabilizing power balances between nations and corporations. The researchers are urging policymakers to increase visibility into how AI research automation is being implemented. This warning adds to a growing list of concerns from mathematicians and industry insiders about existential threats posed by unaligned AI scaling.
decoder