Anthropic warns that it’s very close to RSI capability, aka self-building AI
techcrunch ·
In its revised S-1A filing, AI laboratory Anthropic disclosed that it is nearing the capability for recursive self-improvement (RSI), a threshold where AI can significantly accelerate its own development. The company warns that using AI to build more capable models poses considerable safety risks by potentially outpacing human ability to implement necessary safety and alignment measures. Although Anthropic states that no current models have crossed this RSI threshold, internal estimates suggest they may imminently meet their definition of dramatically accelerated progress attributable to AI. This disclosure highlights the dual nature of RSI as both a promise for unprecedented productivity and a peril for potential loss of control. To mitigate these risks, Anthropic emphasizes its commitment to keeping humans in the loop during frontier model development through its Responsible Scaling Policy. The filing also notes that past advancements included leaps in capability that were not immediately recognized at the time of development. While skeptics view such statements as part of IPO marketing, the rapid growth in Claude’s coding and cybersecurity skills lends credibility to the concern. Ultimately, the article suggests that stakeholders should take the possibility of approaching RSI seriously and consider appropriate governance measures.
techcrunch