My context window is expensive — do I really need to send the whole document, or can I cut it safely?

November 5, 2026 · 5 min read

SentX SI

For a single, well-defined task, usually no — you can cut, and the cost case for cutting is stronger than the usual token arithmetic suggests. A technical report posted to arXiv on 3 November 2026, "You don't need long context" (arXiv:2611.05306), by Peter Belcak and eleven co-authors, supplies a concrete public argument for keeping one request's context short. At the abstract level, three findings matter here. Later tokens are costlier to generate than earlier ones, and on frontier open-weight models the disparity may rise to as much as seven times. Through what the authors call frequent compaction, frontier language models sacrifice little to no task accuracy across a broad range of long-horizon and long-input tasks when the context is kept short. And on long-horizon software-engineering tasks, performance does not collapse even when the model takes at most one action per context — a result the authors use to outline a concept they call semantic state space models, an outline rather than a validated architecture.

Source: You don't need long context (arXiv:2611.05306)

Two scope notes before the method. The report is first-party and unreviewed — 31 pages, 8 figures — so the numbers are the authors' claims, not settled benchmarks, and the abstract does not spell out operationally what frequent compaction means; the mechanism lives in the full text. And the seven-fold figure is an upper bound for frontier open-weight models, not a universal rate you can read off your invoice. The authors' own conclusion is hedged to match: their findings question the need for long-context language models.

Source: You don't need long context (arXiv:2611.05306)

Why cutting pays on both sides

The cost works against you twice when you paste the whole document. You pay for every input token you send, and — per the report's cost finding — generation itself gets more expensive as the context the model works over grows. A 20,000-token paste is not just 20,000 input tokens; it also makes the answer you are paying for costlier to produce. The accuracy fear is the other side, and the compaction result is the counterweight: when a task can be served by a short, dense context, the model often does not miss what you left out.

Source: You don't need long context (arXiv:2611.05306)

The keep-vs-drop method

Here is a working test, offered as an editorial suggestion rather than a finding of the paper. Walk the document section by section and ask three questions in order. First, does the task touch this section? If the request is to summarize the termination clause, the exhibits and the insurance schedules do not touch it. Second, where the task does touch a section, does the model need the exact wording, or would the relevant passage extracted faithfully do? Exact wording — figures, dates, clause text you will quote back — stays in the paste; everything else reduces to the passage. Third, is the content something the model already knows — standard definitions, boilerplate that appears in every contract of its kind, well-established protocol? Send only what is specific to your document.

Whatever survives all three questions is your keep set: the task, the passages it operates on, and the exact figures that must come back verbatim. If you cannot name the reason a section is in the keep set, it is in the drop set.

A worked example

This one is illustrative, not a measured result. Suppose you have a 30-page vendor agreement, roughly 15,000 tokens, and you want a risk read on the termination and liability sections. The keep set is the termination clause, the liability and indemnification clauses, the definitions those clauses lean on, and the payment schedule if termination interacts with it — plausibly 1,500 to 2,000 tokens. The drop set is the exhibits, the boilerplate, and everything the task never touches.

The input side of the bill drops by a factor of seven to ten, at any rate. At an assumed $3 per million input tokens — a round illustration, not a quote — the full paste costs about $0.045 per request and the trimmed context about $0.005; across a hundred requests a day that is a four-dollar-a-day gap, before anything the generation side adds. The accuracy question in this scenario is exactly the one the compaction result speaks to: the task is bounded, the relevant passages are extractable, and nothing in the dropped sections changes what a careful read of the termination clause should say.

Source: You don't need long context (arXiv:2611.05306)

Where cutting is not safe

The method has a hard edge. Do not cut when the task is whole-document in nature — find every clause that mentions X, reconcile two versions, check the text for internal consistency — because the answer depends on material you cannot see until you have read everything. Do not cut when the document is the deliverable rather than the input. And do not rely on a model remembering anything that must remain exact: if a figure has to come back verbatim, it has to be in the context you send, every time.

Cutting also cuts exposure

The same trim is a privacy move, not just a cost move. If you send to a service whose terms state that submissions are not confidential, every token you drop is a token that never enters a training corpus or shared memory. SentX's privacy policy says submissions are not confidential and advises against sending trade secrets, privileged communications, protected health information or financial credentials — so the keep set is also your minimum-exposure set. The two decisions point the same direction.

Source: Privacy Policy

How SentX frames the same decision

Victoria's model card puts the keep-vs-drop rule into one line of intended-use guidance: give the task, the relevant context and the source material — not the whole document, the relevant context. For work that spans conversations, the card's safety net is a portable checkpoint: a record of checked findings and open questions you carry between sessions, so what you dropped from one request can be re-attached to the next. The checkpoint builder formats those notes; it does not verify them. And the caveat the card attaches is the same one this method needs — recall is selective and exact retention is not guaranteed, so details that must remain exact get repeated, not remembered.

Source: Victoria: public model card

The decision, then, is not long context versus short context. It is: does this request touch the whole document? When it does not, cut to the keep set — the savings are immediate, and on the class of tasks the report covers, the accuracy risk is small. When it does, send the whole thing and stop optimizing.

Source: You don't need long context (arXiv:2611.05306)