Why Self-Improving Agents Are Unstable: Result Variance and Dependence on Task Order
Self-improving agents are one of the most promising directions in modern AI. They can accumulate experience during operation and get better with each new task. But behind this attractive idea lies a serious problem: such systems are extremely unpredictable. Even the same agent can show completely different results when run again, and its learning efficiency directly depends on the order in which it encounters tasks.
A recent study presented in the paper "On the Fragility of Self-Improving Agents: Variance, Task Order, and Underspecification" (arXiv: 2608.18066) has closely examined this issue. The authors conducted a re-evaluation of two popular methods based on maintaining a text-based memory bank. Such agents store information about solved tasks and use it as external memory when processing new queries. The mechanism seems simple and logical, but in practice everything turned out to be much more complicated.
The Hidden Noisiness of Self-Improvement
The first thing the researchers discovered was a high level of noise in evaluating such agents. In complex environments with multi-step tasks, the results of the same agent can fluctuate significantly from run to run. And adding a self-improvement loop does not just preserve this noise—it noticeably amplifies it. The paradox is this: the more an agent tries to learn from its experience, the less stable its behavior becomes.
Why does this happen? Partly because the text-based memory bank accumulates information non-deterministically. At each step, the agent chooses what to keep and what to discard, and these decisions depend on many factors. As a result, even with identical input data, the agent's internal state can differ greatly, which is what produces the variance.
Task Order as a Hidden Curriculum
The second important observation concerns the influence of task order. It turned out that agent improvement strongly depends on the sequence in which it encounters tasks. Previous work on self-improving agents often used a fixed "default" order. The researchers found that this order effectively creates an implicit curriculum—tasks are arranged from simple to complex, giving the agent the opportunity to gradually build up competencies.
However, simply shuffling the tasks randomly causes learning efficiency to drop sharply. This means that the reported success of many systems was largely predetermined by a favorable arrangement of material, rather than by the agent's actual ability to learn. In essence, the default order acted as a hidden precondition for success—a factor that was usually not even mentioned in method descriptions.
Underspecification as the Root of the Problem
The authors hypothesized that the main cause of fragility lies in the underspecification of tasks and environments. When task conditions are described insufficiently, the agent is forced to make many ambiguous decisions. This generates randomness in how it accumulates knowledge and in how it later applies it.
To test this hypothesis, the researchers attempted to make the memory construction process more specified. They added detailed evaluation rubrics and feedback from the environment—that is, the kind of information that should reduce ambiguity. The result was telling: additional specification only partially closed the performance degradation observed in the experiments. Significant gaps remained, indicating the existence of other, not yet characterized factors contributing to instability.
What This Means for Practitioners
These findings have direct implications for developing and testing such systems. First, you cannot trust the results of a single run. The agent must be run multiple times, and you should report not only average metrics but also the variance. Otherwise, you can easily draw a false conclusion about a method's effectiveness based on a lucky coincidence.
Second, when comparing different approaches, you must test them on randomly shuffled task orders. If a method only works on a "greenhouse" curriculum, its practical value is questionable—in reality, tasks rarely arrive in a perfectly structured sequence.
Finally, underspecification is not just a feature of specific benchmarks but a fundamental property of many real-world scenarios. Agents have to operate in conditions where the rules are not fully clear, and this will always create a risk of unpredictable failures. Therefore, along with improving the algorithms themselves, it is important to design interfaces and processes that allow humans to effectively control agent behavior and intervene before an error becomes critical.
Instead of a Conclusion
Self-improving agents remain an extremely promising but still immature direction. The study clearly demonstrates that their current implementations are far from reliable. Result variance and dependence on task order are not annoying obstacles but fundamental properties that must be taken into account. Without rigorous evaluation protocols that include multiple runs and stress tests in complex environments, we risk mistaking random successes for real progress. And that is already a question of trust in AI systems as a whole.
