Why Relying on Trace Routing Is Dangerous
In mixture-of-experts (MoE) architecture models, each token passes through a small subset of "specialists." After processing, a formal trace remains—which experts were invoked and with what weights. Such a trace is convenient to use as an explanation, but it resembles a record of phone calls without the content of the conversation: you can see who called whom, but not what they agreed on.
A recent preprint (Chen et al., arXiv:2608.17638) shows that trace-based interpretation misses an important layer of information. The authors propose reading not only the visible route but also the model's internal reasoning state. The key idea is two-level reading: first, a compact semantic space, then a cheap proxy reconstructed from ordinary routing statistics.
Two-Level State Reader
J64: 64 Axes Instead of a Full Vocabulary
The first level is built from space J, whose size is comparable to the model's vocabulary. This space is distilled into a compact semantic frame, J64—just 64 axes. The result is not an arbitrary thematic projection but a set of invariants trained on the model's own reasoning states.
The main benefit of J64 is that it sees states absent from the visible trace. Two identical routing traces can hide different internal processes, and J64 helps distinguish them. In addition, it separates two different loads: how much effort the model invests in reasoning and how difficult the example itself is for it.
Validation on held-out data shows an increase in area under the ROC curve of 0.096–0.135 compared with a baseline approach that reads the same rollout as token occupancy and uses the same aggregation. In other words, the improvement comes precisely from the state-reading method, not from changing the model itself or the data format.
R64: The Same Picture from Cheap Statistics
J64 has a practical drawback: it requires access to the model's internal representations. To make the approach applicable in production, the authors reconstruct J64 from native expert-routing statistics. This proxy is called R64 and introduces no significant overhead during inference.
The median per-axis correlation between R64 and J64 across three models and two architecture families reaches 0.69–0.86. On the gpt-oss-20b model, R64 retains 95–100% of the predictive gain of the original J64. In other words, for many tasks it is not necessary to compute the full semantic space: it is enough to collect statistics from the already existing routing mechanism.
Two Time Resolutions for Test-Time Decisions
State reading works not only as a post-hoc explanation but also as a decision-making tool. The authors consider two scenarios: analyzing completed candidates and managing generation in real time.
Selecting from Ready-Made Options
When the model has already generated several answer variants, J64 and R64 improve the selection of one branch among candidates. Weighted voting was tested separately: if weights from R64 are used instead of simple majority, the result outperforms ordinary majority voting in seven out of eight settings. This means the hidden state can be used as a more accurate signal for answer aggregation.
Stopping and Resampling During Generation
The second scenario is generation where decisions must be made as tokens appear. Sliding reading windows feed into a cumulative "stop and resample" policy: at a certain point, the model halts the current branch and starts generating anew. The operating point of this policy is fixed only on training questions, so the method does not adapt to known answers at test time.
J64 yields an accuracy gain of 1.1–5.9 points over a control experiment using shuffled siblings. R64, built solely on routing, retains 0.9–3.2 points of that gain. Even the cheap proxy proves useful enough for generation control.
Can Reasoning Be Modified Through the Router
The final part of the work goes beyond passive interpretation. The researchers edit the router so as to affect the mechanism corresponding to J64. After this, the model's behavior changes in a predictable way—one that follows from the J64 interpretation. The model's diagnosable "stalling" shifts from numerical guessing to precise symbolic execution.
This is an important signal: the J64 axes capture not just correlation but a causal role in reasoning. If router editing produces the expected changes, then the read state genuinely participates in decision-making rather than being a side artifact.
MoE tracing remains a convenient but overly coarse interface to the model. Two-level reading—first a frugal semantic projection, then a routing-based proxy—allows us to see what happens before and beyond visible expert calls. Such internal "instrument panels" could become the next standard tool for debugging and controlling large models.
