AI performance costs are falling faster than those of any previous technology
decoder ·
Recent research indicates that the cost of achieving fixed performance levels in AI benchmarks has dropped dramatically, with Epoch AI reporting a 47% quarterly decline since 2023. This rate of cost reduction is significantly faster than any previous transformative technology, although it reflects market prices rather than pure algorithmic efficiency. MIT researchers provide a more nuanced view, attributing only about a threefold annual improvement to algorithmic gains after accounting for cheaper hardware and competitive pricing pressures. The disparity between studies arises because newer reasoning models often consume more compute per task, which can increase costs despite lower token prices. While matching past top-tier capabilities has become much cheaper, running current state-of-the-art models frequently costs more due to increased test-time compute. Additionally, benchmark scores may not always correlate with real-world efficiency, as companies might optimize for specific tests without delivering proportional practical benefits. Consequently, selecting an AI model requires evaluating factors beyond simple price-per-token metrics, such as latency, accuracy, and suitability for specific workflows.
decoder