In brief
There is a class of attacks commonly called indirect prompt injection (IPI): a malicious instruction is slipped to the model not through the chat, but through the output of an external tool — a web page, an email, a file, or an API response. The agent honestly reads it as data and may carry out someone else's side task.
A team of researchers from China (Jianshuo Dong, Yiming Liu, Maosen Zhang, Nan Deng, Peng Xu, Xiaoping Zhang, Tianwei Zhang, Jie Zhang, Han Qiu) showed that the moment an agent "gets injected" is already recorded in its internal representations — before it has produced its first token. The work is posted on arXiv (2608.02657, v1 — August 1, 2026, v2 — August 24, 2026), DOI: 10.48550/arXiv.2608.02657, with the code openly available.
The main surprise is not the vulnerability itself, which has been known for a long time, but that the vulnerability signal can be extracted linearly and robustly, even on models with hundreds of billions and trillions of parameters. In other words, this is not a fragile artifact correlation from a single dataset.
IPI exposure: what exactly the probes look for
The authors introduce a working concept, "IPI exposure" — a state of the model during processing that corresponds to an unwanted third-party instruction having entered its context. This is not the same as a successful attack: the model may "notice" the threat and still carry it out. What is meant here is specifically the internal state of exposure.
The key phrase here is "before generation." The probe looks at the hidden states at the input to the decoder, not at an already-produced response. This matters for two reasons: first, the signal is present even where the final behavior looks harmless; second, diagnostics can be done before the model has had time to call a tool with side effects.
The experiment: linear probes on eight models
Scale and coverage
The test ran on eight models. Among them are GLM-5.2 — 753 billion parameters — and Kimi-K3 — 2.8 trillion parameters; results for this model were added in the second version of the paper. Simple linear probes trained on hidden states were used — essentially a linear classifier on top of a vector.
The abbreviation AUROC here means the quality of separating two classes: trained on one dataset, the probe achieved 0.90+ on attacks, agentic instructions, and task sets it had never seen before. In other words, the training was not tailored to a specific attack scenario.
Robustness of the signal
A separate line of experiments in v2 is devoted to generalizability. The probes retain their predictive ability:
- against adaptive attacks, that is, when the adversary tries to fool the probe itself;
- in cross-lingual settings, when the training and test examples are in different languages;
- on new types of tasks and instructions not encountered during training.
It is precisely this triad that is usually the weakest point of any detector based on internal representations, which is why testing it matters more than the AUROC number itself.
The gap between knowing and acting
The most unpleasant finding of the paper is the so-called knowledge-action gap. After post-training, modern models do encode signals that predict IPI exposure, but they do not use them as a reliable trigger for safe behavior. The model "senses" the problem and proceeds anyway.
The reason, judging by the logic of the work, is that the signal exists in the representations but is not connected by a stable link to the action policy. Neither a plain refusal nor instructions in the system prompt close this gap — they operate at a different level.
Probe-driven defense
Since the signal exists, it makes sense to use it directly. The authors propose a defense in which the model's reasoning is activated only when the probe registers exposure. Simply put: the detector fires rarely and accurately, and the expensive analysis procedure is launched on its command rather than always.
On the complex configurations of the AgentDojo benchmark, this approach markedly reduces the share of successful attacks — for example, from 34.6% to zero on Qwen3.5-27B. An important detail: on "clean" tasks, where there is no injection at all, the method preserves utility better than baseline defenses. This is exactly what detectors usually lack — they either catch little or get in the way all the time.
What hidden states "say" in words
The third part of the work is an explanatory framework. The authors look for natural-language formulations that correlate strongly with what the probe registers. The result is a kind of textual profile: they show which particular "thoughts" accompany the detector firing.
What is interesting here is the heterogeneity. In some models, the latent signal corresponds to a direct sense of threat — something like "this text contains an instruction I was not given." In others, it is tied to indirect operational cues: an unusual data format, a suspicious source, a conflict with the current task. That is, the same detector may rely on different internal mechanisms depending on the model.
What this means in practice
For those building agentic systems, the conclusions are quite practical:
- You can check before acting. The signal is available at the input to generation — which means it can be built into the pipeline before the agent pulls the trigger on a tool.
- A linear probe is a cheap layer of defense. It is incomparably lighter than a second pass through the model or an external classifier on every request.
- Don't rely on the prompt alone. The knowledge-action gap means that "please ignore instructions from documents" is not a guarantee, even if the model understands everything.
- Factor in the cost of false positives. The method's main argument is that it does not get in the way on clean tasks, and this is worth testing on your own traffic.
Open questions
The paper answers the question "is there a signal" convincingly, but leaves a few uncomfortable spots. A detector based on hidden states is yet another attack surface: if the adversary knows about the probe, they can optimize the injection so that the signal does not arise. Cross-lingual robustness has been tested, but the language and domain coverage is still limited to the data used in the experiments.
Another question is portability. The probe is trained for a specific model: each architecture has its own representations, and the work does not yield a universal "sensor" that can simply be plugged into any API. For closed models with text-only access, this method is fundamentally inapplicable — hidden states are required.
Nevertheless, the direction looks promising. Instead of arguing about how well a model follows instructions, the researchers propose looking at its internal state and confirming that the needed signal is already there. After that, it is a matter of engineering: how to reliably turn that knowledge into action.
