GPT-6-Astra solves 49 of 50 physics research problems at $2.40 apiece
decoder ·
A new benchmark called TheoPhysEval, developed by thirty-five researchers from institutions including Harvard and Google DeepMind, tests AI agents on 50 unpublished theoretical physics problems. GPT-6-Astra emerged as the top performer, correctly solving 49 of the 50 problems at an average cost of under $3 per issue. The benchmark focuses on routine "turn-the-crank" calculations rather than grand open challenges, revealing that leading models have effectively memorized significant portions of physics literature. While AI now matches or exceeds human experts in executing specific calculations, authors note a lack of scientific "taste" prevents these tools from generating breakthrough theories yet. The study highlights that internet access had minimal impact on performance, suggesting strong internal knowledge bases within current large language models. Additionally, some initial failures were attributed to minor errors in the expert-provided answer keys rather than model deficiencies. Researchers warn that while AI lowers barriers for basic computations, it may eventually skew resource advantages toward well-funded groups in theoretical physics.
decoder