Can I make my agent better by evolving the scaffolding program around a frozen model, instead of retraining it?
5 min read
The scaffolding is a real improvement axis, and a paper posted to arXiv on November 13 treats it as the main character. But there is a gap between your question and what the paper measured that is worth naming up front. ModHarness-RL, by Zelin He, Siyu Zhu, Jacob Helwig and colleagues, evolves the scaffold program while the model is also being trained with reinforcement learning. The strictly frozen model you have in mind, where the weights never move and only the surrounding program changes, is not the configuration they tested. What they tested is co-evolution: a harness that tracks how the model's behavior shifts as the model itself improves.
Source: ModHarness-RL: Recasting Harnesses as Learnable and Evolving Programs for Agentic RL
The paper's definition of harness is precise: the scaffold around the model that decides what the model sees of its own interaction and what guidance it receives. Most agentic reinforcement learning today runs under a hand-written harness that stays fixed throughout training. The authors' premise is that an effective harness depends on the domain, the data and the model, and keeps changing as the model trains, so pinning it down leaves adaptation on the table. And unlike the usual framing, where you tune the policy inside a fixed scaffold or vary one model's outputs while leaving everything else static, ModHarness-RL makes the scaffold itself the thing that learns.
Source: ModHarness-RL: Recasting Harnesses as Learnable and Evolving Programs for Agentic RL
The core move is to make the harness a program. The agent's interaction is recorded as a structured State, a set of typed blocks, and two parts of the harness become programs over those blocks. The Compression program decides how accumulated history is assembled into the model's context. The Steering program decides what guidance the model gets when its behavior deviates. During RL training, candidate programs are proposed from recent trajectories, verified by execution, and replayed over past trajectories, so the harness moves when the model moves.
Source: ModHarness-RL: Recasting Harnesses as Learnable and Evolving Programs for Agentic RL
The empirical scope is three benchmarks: CodeSearch, studied in depth, plus ALFWorld and BIRD SQL, across models from 1.5B to 8B parameters. In most of those settings, not all of them, the learned Compression and Steering programs beat the fixed-harness baseline on held-out performance. The qualifier is doing real work: the improvement is not universal.
Source: ModHarness-RL: Recasting Harnesses as Learnable and Evolving Programs for Agentic RL
The evidence has edges. Models top out at 8B parameters, so if your agent runs on a much larger model, you are outside the measured range. Three benchmarks, one studied in depth. A single version of the paper, no peer review yet. The authors' conclusion, that a harness for agentic RL can be learned from the task's data and evolved with the model, is their stated suggestion, not settled fact.
Source: ModHarness-RL: Recasting Harnesses as Learnable and Evolving Programs for Agentic RL
So does it answer your question? Partially, and what follows is my editorial reading rather than the paper's. The mechanism, a scaffold as a readable program that you can propose changes to, verify by execution, and test against recorded history, is exactly the kind of thing you can run around a model you cannot retrain. The paper measures the loop in the setting where both sides move, and the frozen-model variant sits outside that measured scope. Treat the gain as plausible and directionally supported, unmeasured at your scale, and cheap enough to test.
If you are deciding whether scaffold evolution is your lever, I would run three checks. First, weight access: if you can retrain and you have the data and compute, retraining remains the deeper lever, and scaffold work is complementary rather than a substitute. If you are on an API model or licensed weights, the scaffold is the only lever you actually control. Second, auditability: a program change is a diff, readable, reviewable, rollbackable; a weight update is not. If your agent touches anything where you must explain what changed and why, the program form matters. Third, loop speed: program updates take effect immediately, so a propose, verify, replay, evaluate cycle can run many times in the time one training run takes. If your failure mode is shipping a context template and never touching it again, the cost of trying is low.
Concretely, the pattern you can borrow without the RL machinery: record your agent's interactions as structured blocks; when your held-out task scores dip, propose a change to the compression or guidance rule, with a stronger model or by hand; execute it over a replayed batch of past interactions; keep it only if held-out scores move. That is the paper's loop with the RL inner loop removed, and it is my reconstruction, not their protocol.
There is a first-party echo of this idea worth naming. SentX's model card lists self-learning and self-improvement through experience as current product capabilities of its Victoria model, behavior that improves without the user touching weights. That is SentX's own disclosure about its own product, and the card does not publish a measured improvement rate, so it stands as an owner-confirmed capability in the same territory, not as independent validation of ModHarness-RL. The overlap is methodological: behavior that improves through experience and interaction, without the user touching the weights.
Source: Victoria AI Model Card
The bottom line: the axis is real, the measured gains are positive in the range tested, and the frozen-model version is not yet measured. The cheapest next step is a replay-and-diff loop over your own recorded trajectories. If it moves your held-out numbers, you have found an improvement the retraining camp cannot offer, one you can read, review and roll back.
Source: ModHarness-RL: Recasting Harnesses as Learnable and Evolving Programs for Agentic RL
Sources
- ModHarness-RL: Recasting Harnesses as Learnable and Evolving Programs for Agentic RL — arXiv
Supported claims and limits
- ModHarness-RL, arXiv:2611.18533, by Zelin He, Siyu Zhu, Jacob Helwig, Yujian Liu, Li An, Rohit Jain, Shiyu Chang, Alborz Geramifard, submitted 13 Nov 2026; harness is the scaffold around the model that decides what it sees of its own interaction and what guidance it receives
- Most existing approaches train under a hand-written harness that stays fixed; an effective harness depends on the domain, data and model and keeps changing as the model trains
- Interaction recorded as structured State of typed blocks; Compression program assembles accumulated history into context; Steering program supplies guidance when behavior deviates; candidates proposed from recent trajectories, verified by execution, replayed over past trajectories
- Scope: CodeSearch studied in depth, ALFWorld and BIRD SQL analyzed further, models 1.5B to 8B; Compression and Steering improve held-out performance over the fixed-harness baseline in most settings, not all
- Single v1, submitted Fri 13 Nov 2026 18:45:49 UTC; authors' stated suggestion that a harness for agentic RL can be learned from the task's data and evolved with the model; no measured frozen-weight variant
- Victoria AI Model Card — SentX SI
Supported claims and limits
- Victoria self-learns and self-improves through experience and interaction; the card classifies these as current product capabilities
- First-party product disclosure, not independent certification; card does not assert a verified model size, training-data inventory or benchmark score, so no measured self-improvement rate is available to quote