AI models' written reasoning steps correspond to distinct internal patterns, a new study finds
decoder ·
A new study from South Korea's KAIST and Naver AI Lab tested whether the reasoning steps a language model shows in its text output can also be identified in its internal numerical representations. The researchers defined eight recurring reasoning operations, including extraction, decomposition, formula recall, deduction, and computation, and had Qwen2.5-7B, Qwen3-8B, and Gemma4-31B solve math problems, split solutions into segments, and use GPT-5 to label each segment. They found the operations are reliably separable inside all three models, with the clearest signal in the middle layers. A classifier based only on word choice performed worse than one analyzing internal representations, and position in the solution path did not explain the effect, indicating the internal states carry step-type information beyond surface wording. Common function words appeared across different reasoning steps but separated by surrounding operation in middle and later layers, so the same word gets a different internal representation depending on the step. Blocking attention to the preceding 30 tokens weakened the signal, suggesting reasoning steps build on preceding context. Even on incorrectly solved problems, the type of step remained identifiable, and the separability replicated with Llama-3-8B and transferred to GPQA-Diamond and MATH-500 for Qwen3-8B. The experiments are limited to math tasks and a handful of models, and whether the findings can be used to catch errors or steer a model mid-generation remains open; the article says the relationship between text output and internal computation matters for AI safety, as reading chain of thought is an oversight tool but Anthropic found models disclose used hints in only 25 to 39 percent of cases, and OpenAI's Astra uses Recurrent Depth to shift part of reasoning into internal numerical representations.
decoder