Why LLM Overconfidence Is Not Just a Bug
Modern language models have learned to express thoughts smoothly and persuasively. But behind this persuasiveness often lies what experts call hallucinations: the model produces plausible-sounding but completely fabricated content. To the user, this looks like even, confident text — and that is exactly why the problem is dangerous.
Most familiar hallucination detectors work crudely. They evaluate the entire response as a whole or individual sentences, reporting something like "there seems to be an error here." But to understand exactly where the model "made up a fact," a finer-grained check is needed — at the level of individual tokens. Only token-level detection allows localizing the false fragment and intervening in generation precisely, without touching the rest of the text.
MoE Models Accidentally Provide a Clue
In architectures like Mixture-of-Experts (MoE), a curious principle is at work: during a single forward pass, only a sparse subset of "experts" is activated, not the entire network. The choice of whom to call is made by the routing mechanism. And it is precisely at this moment that internal signals arise that dense architectures simply do not have:
- router entropy — how "uncertain" the model is about which expert to consult;
- expert disagreement — how strongly their intermediate results diverge;
- expert usage patterns — which specialists are more often activated.
Previously, these signals were hardly used for hallucination detection. Researchers João Fonseca, Rodrigo Rodrigues, and Paolo Romano in the paper arXiv:2608.17687 showed that this was a missed opportunity: hidden indicators can reveal the moments when the model starts to fabricate. The work was posted on arXiv on August 18, 2026.
InnerExpert: A Detector That Looks Inside the Model
The method InnerExpert proposed by the authors combines two types of data: MoE-specific routing-level signals and standard internal transformer representations. All of this is collected into compact feature vectors for each token. A lightweight classifier then processes these vectors and determines whether a token is a hallucination.
The key feature is automatic training. Labels for training the detector are produced by the LLM-as-a-judge pipeline, where one language model evaluates the output of another. No manual annotation is required. This means the detector can be quickly adapted to new versions of generative models by simply re-running the pipeline.
What the Tests Showed
The experiments covered five datasets and two different MoE architectures. In the tests, InnerExpert consistently outperformed existing detectors. The best results reached 0.91 AUROC at the whole-response level and 0.76 AUROC at the individual-token level. Moreover, a single forward pass is sufficient — no additional verification models or repeated generations are needed.
A high answer-level result is useful for coarse filtering. But the truly valuable metric is token-level: it allows pinpointing false fragments of text and, for example, rewriting them or highlighting them for the user. This is no longer "somewhere there's an error," but a specific place in the response.
Why This Matters
Hallucinations remain one of the main barriers to deploying LLMs in medicine, law, finance, and other error-sensitive domains. The ability to look into the model's internal signals and see its "doubts" in advance is another practical step toward transparent generation. Hidden indicators do not completely solve the problem, but they provide a working tool that is already available.
It is also interesting that hallucination diagnostics were born from the peculiarities of the architecture itself. MoE models were created for speed and efficiency, and their internal structure unexpectedly turned out to be useful for quality control. Further research will surely go even deeper — toward finer signals within experts and routing mechanisms.
