What this article is about
When a large language model solves a complex logical or mathematical problem, the generation process is not just a sequential output of tokens. Each step relies on an internal state in which the model holds the condition, intermediate results, and the connections between them. Strong reasoning LLMs use this space effectively, but even they experience failures. The problem is often not a lack of knowledge, but rather that the necessary cognitive mechanisms—self-checking, reformulating the problem, returning to previous conclusions—are not activated in time.
New research, the first version of which appeared on arXiv in May 2026, with the final version in August, was accepted at the EMNLP 2026 conference. The authors presented the Latent Reward Steering (LRS) framework. This is an adaptive inference-time method: it does not change the prompt text or add instructions to it, but instead acts on the model's internal representations. In essence, the system learns to "fix" the reasoning process at the moment it sees it going off track.
Why behavioral prompts are not enough
The most common way to improve LLM reasoning is to explicitly ask it to behave in a certain way: "break the task into steps," "check yourself," "think first, then answer." Such techniques control behavior at the external, textual level. They assume there is a universal scenario that is equally useful for all errors. But in practice, failures can be completely different: the model may misunderstand the condition, lose an important detail, choose an unsuitable method, or make a calculation error at the last step.
Moreover, different models—and even the same model in different attempts—make mistakes in different ways. A pre-defined prompt does not account for the type of task, the current state of reasoning, or the specific characteristics of a particular LLM. In some cases, it helps; in others, it proves useless, and sometimes even harmful, because it forces the model to mechanically follow a template instead of adapting to the actual line of thought.
How LRS corrects the reasoning process
Instead of giving the model textual recommendations, LRS works at the level of hidden states that the model generates during reasoning. These states carry not only the content of the steps, but also subtle signals of how confidently the model is moving toward the answer. To make such signals accessible for control, a sparse autoencoder is used.
The SAE decomposes the model's dense activations into a small number of interpretable components. Some of these components correspond to useful cognitive behaviors, such as intermediate verification or revising a conclusion. It is precisely these that LRS targets. This approach allows influencing the model's "thinking" more precisely than is possible with verbal instructions.
Training the latent reward model
To distinguish successful internal states from unsuccessful ones, the authors train a separate reward model. It takes as input not the final answer, but intermediate latent states, and evaluates how much they bring the model closer to the correct result. Full reasoning traces labeled by answer correctness are used as training data. This way, the reward model learns to predict in advance whether the current segment of the path will lead to success.
Correction via reward gradient
When LRS detects a state during generation that looks unpromising, it computes the gradient of the reward signal with respect to the components of that state. The gradient indicates the direction in which the latent state needs to be shifted to increase the likelihood of a correct continuation. A key feature of LRS is that this direction is not predefined and is not derived from fixed "rules of good reasoning." It is computed on the fly for each specific state, so the method adapts to the model, the task, and the specifics of the current error.
Reward and confidence gate
Intervening at every step would be too risky: constant external influence could disrupt natural generation. Therefore, LRS uses a reward and confidence gate. It allows correction only for those states that the reward signal marks as fragile. If the current state is assessed as confident, no intervention occurs. This selective approach reduces side effects and preserves the naturalness of reasoning when the model is already coping on its own.
What the experiments showed
The authors tested LRS on several reasoning LLMs used as backbones and on a set of standard benchmarks. The method was compared with various baseline approaches, and in most configurations LRS provided a consistent improvement in accuracy. Importantly, the positive effect was not a one-off case: it was reproduced when changing both the model and the type of tasks.
Post-hoc analysis of the results showed that the improvements do not boil down to simply "lengthening" answers or changing the generation style. The researchers found that LRS implicitly promotes the cognitive behaviors that textual prompts usually try to elicit. The model more often returns to erroneous intermediate conclusions, analyzes conditions more carefully, and demonstrates a more consistent line of reasoning. In other words, the method truly works through internal states, not through external templates.
Why this matters
LRS is interesting not only for its specific results, but also for the shift in the very principle of controlling reasoning. Instead of searching for the ideal prompt wording, the researchers propose learning to gently "correct" the model in the space of its own representations. This approach is potentially more universal: it does not require describing all possible cognitive scenarios in advance and itself finds which aspect of thinking needs correction at a given moment.
For now, LRS is a research framework, not a ready-made product. The authors have open-sourced the code so that other teams can reproduce the experiments and adapt the method to their own tasks. But the direction is already clear: controlling LLM reasoning is gradually moving from the level of words to the level of the hidden mechanisms that determine how exactly the model thinks, not just what it says.
