Google Researchers Use AI’s Search History to Cut the Cost of Self-Improvement
aiinsider ·
Google researchers have developed Dream-RSI, a framework that leverages an AI agent's search history to reduce the costs associated with automated discovery and self-improvement. Instead of retraining the underlying model, the system records previous searches as trees containing outcomes and computational costs, allowing agents to test alternative exploration strategies in a simulated replay environment. This approach enables the selection of more efficient policies for future live searches without repeating expensive experiments. In tests across eight tasks involving algorithm engineering, mathematical optimization, and GPU programming, Dream-RSI matched or exceeded competing methods while significantly reducing the number of required AI-agent calls. For instance, in statistical algorithm design, the system achieved faster runtimes using fewer than half the agent calls compared to fixed exploration policies. The study highlights that improving how an AI allocates resources during a search can be more effective than relying on high-level written guidance from past attempts. However, the method is limited by its inability to predict outcomes for unexplored branches, though it offers a practical path toward more efficient recursive self-improvement.
aiinsider