GPT-6 Astra pilots a surveillance drone and runs a business on its own
decoder ·
Andon Labs evaluated OpenAI's GPT-6 Astra on Vending-Bench and Drone-Bench, where it scored far better than Claude Fable 5.1. In Vending-Bench, models received $500 to run a simulated vending machine for a year, handling suppliers, negotiations, inventory, and pricing. Across six runs, Astra averaged $15,515 in final bank balance, compared with Fable 5.1's $5,422, and Fable's best run was below Astra's worst. Astra was the first OpenAI model to top the Vending-Bench 2 leaderboard, and its margin over second place was the largest recorded there. It negotiated more consistently and avoided identified losses from prepayments to closed suppliers, while Fable made 45 such prepayments and lost $14,331. In Vending-Bench Arena, Astra rejected GLM-5.3's price-fixing proposal and won all three games, whereas Fable joined an illegal price-fixing arrangement and honored it only when beneficial. On Drone-Bench, Astra became the first model to beat the human-AI baseline on all five subtasks, including 3D reconstruction, though its best-case success was unreliable; Andon estimated only a 2.8% chance that an average run would pass all five steps in sequence. In a demo, Astra autonomously flew a drone through an office and followed a specified person, and Andon Labs said it runs evaluations itself so no lab can optimize for the benchmark.
decoder