An agent instead of a standalone optimizer: a unified framework evolving across multiple tasks simultaneously

What Is a Harness and Why Optimize It

A harness is the code around a language-model-based agent. It organizes prompts, tool calls, context management, and execution. The agent’s approach to solving a task depends on how this code is designed.

In most existing methods, a human creates the harness. A separate proposer modifies the solver’s harness. A separate harness evolves for each benchmark. This approach ties improvements to a single task.

Practical criterion: a single framework evolves across multiple tasks at once, rather than separately for each benchmark.

A Unified Framework: The Agent as Its Own Optimizer

Qiankai Xu’s paper was submitted on September 29, 2026. It is titled “Self-Evolving Harness on Multiple Tasks with the Agent as Its Own Optimizer.” The approach is built around a single frozen model and a single version of the harness.

The model first solves tasks as a solver. Then it also acts as the proposer. It reads full run logs and directly edits the harness that runs it. No separate proposer is needed.

Practical criterion: an agent that reads its own runs and edits its own harness replaces a separate optimizer.

Five Benchmarks and Data Splits

Each evolution batch includes tasks from five benchmarks across different domains. Training and held-out tasks are strictly separated.

In addition, evaluations are conducted on five out-of-distribution benchmarks. These benchmarks were not used during evolution.

Practical criterion: out-of-distribution evaluations test transfer, not just adaptation to familiar tasks.

Two Stages of Evolution

The evolution process is presented as training a deep model in two stages:

1. Multitask pretraining.
2. Continued training.

The initial version of the harness consisted of 49 lines. After the first stage, the average score increased by 4.48 points on in-distribution benchmarks. On out-of-distribution benchmarks, it increased by 12.64 points. On the former, it outperformed Codex; on the latter, it matched it.

In the second stage, further evolution on Claw-Eval raised the score on that benchmark from 66.17 to 68.06. The approach outperformed Codex on this benchmark.

Practical criterion: pretraining broadens transfer, while continued training further improves results on a specific benchmark.

Mechanisms That Emerged During Evolution

The authors analyze three mechanisms:

• output truncation;
• history compression;
• independent verification.

Output truncation limits the response length. History compression reduces the context. Independent verification adds a check on the result. These mechanisms emerged from harness edits rather than being specified by a human.

Practical criterion: the mechanisms that emerged provide a reference point for evaluating your own framework.