How many candidate actions should my agent sample per step — and when does best-of-N stop paying for itself?

November 6, 2026 · 6 min read

SentX SI

The short answer, from the most recent published measurement of this knob: start at four. Two candidates does almost nothing, four captures most of the available gain, and eight only pays off when your generator is the weak link in the pair — and even then, you should verify it on your own tasks before committing the budget.

Source: On Turn-Level Best-of-N Sampling for Scaling LLM Agents; On Turn-Level Best-of-N Sampling for Scaling LLM Agents (full text)

The measurement comes from "On Turn-Level Best-of-N Sampling for Scaling LLM Agents," posted to arXiv on November 4, 2026 by Jalaj Bhandari, Dhawal Gupta, Hongli Zhan and colleagues at the Institute of Foundation Models (IFM) at MBZUAI Labs. It targets a specific dial that trajectory-level best-of-N, DFS, MCTS and trained-critic approaches do not directly tune: not which model to call, and not which decoding settings to use on a single call, but how many candidate actions to draw from one generator at each step of an agent run, and whether a second model is worth paying to pick among them. The authors describe the results as preliminary, and the cost side is where the caveat lives.

Source: On Turn-Level Best-of-N Sampling for Scaling LLM Agents

What the paper tested

At each turn, a generator model samples N candidate actions (temperature 1.0, top-p 0.95); a selector model — a stronger LLM — sees the whole list plus the task context and returns the index of the best candidate; only that action is executed. No sandbox is needed for the unchosen candidates, which is what makes this different from trajectory-level best-of-N, where N full runs must be scored after the fact. The authors frame the method as a sampled version of one-step policy improvement, with a formal condition (Proposition 2) under which the induced policy beats the generator policy.

Source: On Turn-Level Best-of-N Sampling for Scaling LLM Agents (full text)

The test ran on Terminal-Bench v2.1 — 89 long-horizon terminal tasks — with the mini-SWE-agent harness and four trials per configuration. The generators were Qwen3.6-35B, at about 47% pass@1, and K2-Horizon-375B, at about 73%. The selectors were deliberately stronger: Kimi-K3, at about 88%, and DeepSeek-V4.1-Flash, at about 82%. Selection was listwise — the whole candidate list in one call — which the authors note is cheaper than pairwise tournament scoring, leaving pointwise and pairwise approaches as future work.

Source: On Turn-Level Best-of-N Sampling for Scaling LLM Agents (full text)

Where extra samples help

N=2 came out very similar to the N=1 baseline, so the first meaningful width is four. At N=4, Qwen3.6's pass rate rose from 47.5% to 52.8%, and K2-Horizon's from 73.3% to 77.9%. At N=8, the two generators diverge in the way the paper's argument predicts: Qwen3.6 kept climbing to 55.2%, while K2-Horizon fell back to 76.7% — below its own N=4 result. The gap between generator and selector capability is the variable that decides whether the extra width pays. The authors are explicit that gains are capped by the base model's capability; how they would be capped by a weaker selector was not tested, because no weaker-selector runs exist in the paper.

Source: On Turn-Level Best-of-N Sampling for Scaling LLM Agents (full text)

The gains also show up in consistency — pass@k coverage improves at small k, and tasks solved across all four trials hold above baseline — and in behaviour. The selector picks work-doing actions (editing a file, testing the last edit, running a check) far more than a uniform draw would, and inspection-only actions drop. Looping falls by 14 to 18 percentage points; retrying a failed command without diagnosing it falls from 17% to 6% for K2-Horizon and from 44% to 22% for Qwen3.6. Each generator's characteristic failure gets fixed differently: for K2-Horizon, context-exceeded and agent-error endings collapse (4.2% to 0.1% and 2.2% to 0.7%); for Qwen3.6, wrong submissions fall from 44.9% to 34.6%. And the improvement is not just a selection effect on successful runs: on failed trajectories alone, bad behaviours flagged by an LLM judge (DeepSeek-V4.1-Flash-1017, 28 rubrics, 36 hard tasks, 360 trajectories, significance at p<0.05) drop from 3.9 to 2.7 per trajectory for K2-Horizon and from 8.9 to 6.8 for Qwen3.6. A failed attempt under selection is more often a real capability gap than a stall in a loop. As a sanity check, the authors verified that reward hacking is rare and does not grow under best-of-N.

Source: On Turn-Level Best-of-N Sampling for Scaling LLM Agents (full text)

What it costs

The price is substantial, and the paper is candid about it. Total output tokens per attempt run from 3.6x the baseline (K2-Horizon with Kimi-K3 at N=4) to 8.7x (Qwen3.6 with DeepSeek-V4.1-Flash at N=8); the authors' own headline is an average of about 3x to 6x. That multiplier already bakes in the biggest offset: solved tasks take about 40% fewer turns at N=4 and about 50% fewer at N=8, because unproductive turns get filtered out before they are spent. What the multiplier does not cover is wall-clock time — attempts hit the shared 8-hour time limit more often under selection, which the authors note is expected — and the selector's own reasoning load, which is why the weaker generator's selector token counts are the larger ones.

Source: On Turn-Level Best-of-N Sampling for Scaling LLM Agents (full text)

When it stops paying

Here is the honest boundary of the paper: it gives you the shape of the curve, not a break-even number. The shape is flat at two, steep at four, and at eight it depends on the generator — for a strong generator, the marginal step is negative. Whether the marginal gain covers the marginal cost is an economic question the paper deliberately leaves to the reader, because it depends on your token prices, your task difficulty, and what a solved task is actually worth. The authors call the approach "computationally very expensive," and their stated open directions are exactly the cost-cutting moves: selecting on only a few turns rather than all of them, and trained critic selectors instead of a frontier LLM. The concurrent work they cite — Kang et al., arXiv:2609.39982, on smaller 4B–9B generators — went the second route, distilling a small selector model from frontier-model traces.

Source: On Turn-Level Best-of-N Sampling for Scaling LLM Agents (full text)

A pilot to find your number

The following is a proposed procedure for sizing the knob on your own workload. It is an editorial suggestion, not a finding of the paper.

Step one: pick a few dozen tasks representative of your production work — on the order of the 36-task hard set the paper used for its behaviour analysis.

Source: On Turn-Level Best-of-N Sampling for Scaling LLM Agents (full text)

Step two: run the N=1 baseline for a few trials per task; record the pass rate and the average output tokens per trajectory.

Step three: run N=4 on the same tasks and the same trial count; compute the pass-rate delta and your token multiplier.

Step four: price it. If the pass-rate gain, multiplied by what a solved task is worth to you, exceeds the added token cost, keep N=4 and repeat the test at N=8. Stop at the N where it no longer covers.

Step five: expect the N=8 step to fail the test if your generator is already strong — K2-Horizon's numbers are the published instance of exactly that.

Source: On Turn-Level Best-of-N Sampling for Scaling LLM Agents (full text)

Step six: if N=4 passes but the budget is tight, the authors' own suggested cut — selecting on a few turns rather than every turn — is the first place to look, though it is unmeasured in the paper.

The verification habit, moved inside the loop

SentX's own model card lists, under known limits, a habit the company asks readers to keep: answers can contain incorrect facts, so open the cited source and check the supporting passage. That is a per-answer verification habit. Turn-BoN is the same habit applied per action — every candidate is checked against the task before it is executed, not picked on confidence. The Qwen3.6 result is the sharpest instance: the baseline agent checked what it had built rather than what the task asked for, and nearly half of its attempts ended in wrong submissions; with a selector rating each candidate against the task requirements at every turn, wrong submissions fell to about a third. Read as an editorial lens rather than a claim of the paper: the value of the verification habit does not depend on who performs it — a human reader, or a stronger model standing in for the agent.

Source: On Turn-Level Best-of-N Sampling for Scaling LLM Agents (full text); Victoria AI Model Card