Google researchers find a way to keep self-improving AI agents from memorizing their tests
decoder ·
Google researchers have developed a new method called RRSI to prevent self-improving AI agents from memorizing their test tasks. This approach addresses the issue where agents overspecialize on limited benchmarks, leading to poor performance on unseen challenges. RRSI regulates the optimization loop by shrinking edit budgets over time and employing a strict critic to reject benchmark-specific tricks. The system prioritizes measurable performance gains over increased compute costs and removes components that no longer contribute to effectiveness. Testing on eight diverse benchmarks showed that RRSI improved generalization to new tasks while using fewer tokens than unregularized versions. Notably, harnesses optimized with this method also enhanced the performance of weaker models without requiring modifications. The study concludes that regularized feedback is essential for reliably increasing agent capabilities without overfitting.
decoder