Do I need my model to reason on every request — and how do I decide when the extra thinking is worth paying for?
5 min read
No. If your bill includes reasoning tokens on every request, the more useful question is not how to make the model think better on the hard ones, but whether most of your traffic buys anything from the pass at all. A paper submitted to arXiv on 12 November 2026, The Value of Thinking: Learning When to Reason in Hybrid Decision Models (arXiv:2611.16961), by Zhengxu Yu, Ke Xu, Meng Fang, Haitham Ammar, Weilin Luo and Jun Wang, takes exactly that question seriously, and it hands you a framing you can carry into your own budget even if you never touch the model.
Source: The Value of Thinking: Learning When to Reason in Hybrid Decision Models
What the paper does
The authors call it the zero-or-T paradigm. Rather than deciding how much to think, the model picks between two modes. In the fast mode the answer comes from the implicit thinking already present inside a single forward pass, and no reasoning token is generated. In the slow mode the model runs a full autoregressive reasoning pass at a cost of T tokens. One frozen backbone serves both modes; what decides between them is a lightweight self-routing block.
Source: The Value of Thinking: Learning When to Reason in Hybrid Decision Models
The block does not run on a rule table. It estimates the value of thinking — the utility the slow pass is expected to add over the fast one — from what the implicit thinking has already produced, and it triggers the slow pass only where that estimated value exceeds λ, a price set per request. That is the whole decision, reduced to one line: is the expected gain worth the price, and what is the price?
Source: The Value of Thinking: Learning When to Reason in Hybrid Decision Models
Two details make it practical. First, existing adaptive-reasoning methods either shorten the reasoning a model writes or skip it by switching modes, but in either case the answer is still written out token by token; the fast path here skips generation entirely, which is where the latency win comes from. Second, in agentic work there is rarely a label for whether any particular intermediate decision needed thinking, so the authors introduce Calibrated Decision Policy Optimisation (CDPO), a reinforcement-learning recipe that trains the routing block from episode-level returns rather than from per-decision labels.
Source: The Value of Thinking: Learning When to Reason in Hybrid Decision Models
What the results show
HDM is evaluated on MMLU-Pro, AGIEval, GPQA-Diamond, JevBench, BANKING77 and MiniWoB. The abstract reports two headline results, and because they come from different model sizes on different benchmarks they are two data points, not one. On JevBench, HDM-4B scores 0.857 against 0.851 for the baseline the paper names Jev — +0.7% — at a median latency of 0.179 seconds against 0.652 seconds, a 72.5% reduction. In the agentic venue MiniWoB, HDM-27B cuts token cost by 80.1%.
Source: The Value of Thinking: Learning When to Reason in Hybrid Decision Models
Read them at that scale. On one 4B model and one benchmark, gating buys a small accuracy gain and a large latency cut; on one 27B model in one agentic environment, it cuts token spend by roughly four-fifths. The pattern — most of the traffic does not need the pass — is what the method is built around, but each number stands on its own model and benchmark.
Source: The Value of Thinking: Learning When to Reason in Hybrid Decision Models
How to set the price: an editorial suggestion, not a finding of the paper
The paper gives you the decision rule; setting λ is your job. Here is a way to do it from your own ledger, proposed as an editorial method rather than measured anywhere:
- Measure your T. Log how many reasoning tokens a full pass actually takes in your workload, not the headline figure.
- Price the pass. Multiply by your per-token rate — output and reasoning tokens if your provider prices them differently — and add the latency cost the pass imposes. If you carry a response-time budget or pay in user wait, the seconds matter as much as the tokens.
- Use that total as your starting λ. It is the per-request cost of thinking, stated in the units you already manage.
- Tune against your own quality metric. Start high — let thinking run rarely — and lower λ until the quality you measure on your own task set stops improving. The MiniWoB figure suggests real headroom in agentic traffic, but that is their environment; your mix of easy and hard requests will set your own number.
What this decision is — and is not
It is easy to blur this with three adjacent knobs. Model tiering routes a request across different models. Best-of-N asks how many candidates to generate per step. Step-level resampling asks where in a chain to redo work. Zero-or-T is narrower and cheaper to reason about: on one frozen model, are any reasoning tokens generated at all? If the answer turns out to be no for most of your traffic, you have cut a line item before you ever touch the other three.
The same discipline at the service level
Naming the price of each unit of compute before you spend it is not confined to model training. SentX's own model card does the service-level version: availability, daily limits and media allowances are scoped per service and plan, so what each unit costs is stated before you use it. That card is a first-party product disclosure from SentX SI, not an independent certification, and it claims no per-request reasoning gating of its own. The transfer is methodological: a budget you can read beats a default you inherited.
Source: Victoria AI Model Card | Capabilities, Evidence and Limits | SentX SI
If reasoning tokens are on every line of your bill, the move is not to think harder on the hard requests. It is to put a price on the pass, find out how often that price is worth it, and let the number — not the default — decide.
Sources
- The Value of Thinking: Learning When to Reason in Hybrid Decision Models — arXiv
Supported claims and limits
- Zero-or-T paradigm: one frozen backbone, fast mode generates no token from implicit thinking in a single forward pass, slow mode spends T tokens; lightweight self-routing block triggers slow mode only where estimated value of thinking exceeds per-request price λ
- CDPO is a reinforcement-learning recipe training the routing block from episode-level returns where no per-decision label exists, as in episodic agentic tasks
- Evaluation set is MMLU-Pro, AGIEval, GPQA-Diamond, JevBench, BANKING77, MiniWoB; the abstract reports headline figures only for JevBench and MiniWoB
- On JevBench HDM-4B scores 0.857 vs Jev's 0.851 (+0.7%) at median 0.179s vs 0.652s (−72.5%); single 4B model, single benchmark, baseline named Jev not further defined
- On MiniWoB HDM-27B cuts token cost by 80.1%; different model size and agentic venue, not comparable to the JevBench figure
- Submitted 12 Nov 2026, cs.AI, 20 pages, 3 figures; authors Yu, Xu, Fang, Ammar, Luo, Wang
- Victoria AI Model Card | Capabilities, Evidence and Limits | SentX SI — SentX SI
Supported claims and limits
- Availability, daily limits and media allowances depend on the current service and plan; the card is a first-party product disclosure, not an independent certification, and makes no claim of per-request reasoning gating