No AI agent scores above 12 percent on SemiAnalysis' new real-world work benchmark
decoder ·
SemiAnalysis has released the GDPx benchmark, which evaluates AI agents on 206 tasks within realistic professional software environments rather than simple knowledge quizzes. The results indicate that no current model performs well, with the highest score being just 12 percent achieved by Claude Opus 5.5. Google's Gemini 4 Carbon followed closely at 10.9 percent while offering significantly lower costs, and GPT-6 Astra scored 9.6 percent. Critics argue that safety filters heavily distort these rankings, particularly in cybersecurity and biology domains where some models received zero scores. Additionally, concerns have been raised about the secrecy of the test tasks and the unequal conditions created by each provider's proprietary agent software. Despite these criticisms, SemiAnalysis aims to update the benchmark regularly to track progress in real-world work simulation.
decoder