OpenAI calls for international standards on AI that could improve itself
decoder ·
OpenAI is advocating for the establishment of international standards regarding advanced AI development, with a specific focus on recursive self-improvement (RSI). This technology allows AI systems to automate the research required to create subsequent generations of AI, potentially accelerating progress at an unprecedented rate. The company emphasizes that fully autonomous RSI does not currently exist and should only be pursued once adequate safety measures are in place. Without mature safeguards, there is a risk that humans could lose control over AI development, leading to systems that are less aligned and more dangerous. To address these concerns, OpenAI proposes that the United States lead the creation of global technical standards by leveraging existing national institutes and organizations like CAISI and ISO. These standards would include shared measurement methods, incident reporting protocols, and rules for human oversight of automated research. A recent UN scientific panel has echoed these concerns, warning about the potential loss of control over autonomous agent swarms. Consequently, the UN has suggested international cooperation frameworks similar to those used in aviation, nuclear energy, and cybersecurity.
decoder