How do I make a long-running agent pull the right memory at decision time — not just the latest one?

October 31, 2026 · 5 min read

SentX SI

The selection rule to install is marginal value, not recency: bring a memory into context when the information it adds exceeds the cost of reaching it, and leave out what is redundant even if it is individually useful. That rule is the core of a paper Daniel Strauß, Zhenshan Bing and Alois Knoll submitted to arXiv on 29 October 2026 (arXiv:2610.40545) and that was accepted at the NeurIPS 2026 Workshop on Long-Context Foundation Models: Retrieval as Traversal: Value-Indexed Memory for Stateful Long-Context Agents. One caution up front, because it shapes everything below: the paper's only experiment is a proof of principle on a toy environment, so the mechanism is a well-argued design and the number attached to it is a single data point.

Source: Retrieval as Traversal: Value-Indexed Memory for Stateful Long-Context Agents

The problem the authors start from is the repetition built into current long-context agents. Attention or global retrieval can retain large stores, but each forward pass again searches for what matters. Recurrent compression avoids the search only by deciding earlier what information to preserve. Neither mechanism directly exploits the property the paper builds on: useful memories at neighbouring retrieval steps may substantially overlap. An agent that just pulled one fact about a project is likely to need the next fact nearby in some organisation of the store, not somewhere it has to re-find from scratch.

Source: Retrieval as Traversal: Value-Indexed Memory for Stateful Long-Context Agents

The mechanism

The proposal makes retrieval a persistent local traversal through an address space. The agent keeps its position across steps instead of restarting a global search, and two things are separated: a memory's content and its address. Utility is defined by the information revealed by retrieving the content; geometry determines only how cheaply that information can be reached. From that split comes a cost-regularised indexing objective that preserves information value inside local neighbourhoods while charging for every memory made locally accessible.

Source: Retrieval as Traversal: Value-Indexed Memory for Stateful Long-Context Agents

Its marginal-value form produces two pressures working in opposite directions. The attractive pressure says complementary memories should be co-retrievable when the additional value exceeds retrieval cost — the useful neighbour gets pulled along. The exclusion pressure says redundant memories should not occupy the same retrieval set solely because they are individually useful — the loud duplicate gets left behind. Together they do what a recency ranker cannot: they select by what a memory adds now, relative to what is already reachable.

Source: Retrieval as Traversal: Value-Indexed Memory for Stateful Long-Context Agents

What the single result shows

As a proof of principle, the authors test only the simplest fixed-geometry instance, on MiniGrid Memory. With a five-token context, a persistent local traversal head reaches 0.95 mean return after 0.59 million training steps, compared with 0.94 million for Gaussian absolute addressing and more than two million for categorical absolute addressing. The comparison measures how quickly each method reaches that return in one small task — a learning-speed result, not a capability leaderboard. The abstract is explicit about what the experiment does not cover: it does not test learned geometry or repeated retrieval. The takeaway it frames is a hypothesis — that stateful retrieval may be easier to learn than stateless — and the authors state that testing the full proposed method requires further experimentation.

Source: Retrieval as Traversal: Value-Indexed Memory for Stateful Long-Context Agents

Checking the numbers against the abstract

The table below maps each figure and claim in this article to the abstract's own wording, with its qualifiers intact. It is the fastest way to audit the retelling.

Source: Retrieval as Traversal: Value-Indexed Memory for Stateful Long-Context Agents

A probe you can sketch for your own agent

The following is an editorial suggestion adapted from the paper's marginal-value logic, not a measured finding; nothing here reports it tested on any product, including Victoria. The point is to give the rule a shape you can try.

1. Choose one long-running scenario — a multi-session project, a support queue, a research log — where your agent acts on remembered facts.
2. For each stored entry, record two fields: a last-use value, how useful the entry proved when last retrieved, and an access cost, how far it sits from what the agent is currently working on (session distance, topic distance, or whatever proxy you maintain).
3. At decision time, retrieve entries whose marginal value — the information they add beyond what is already in context — clears a cost threshold, instead of taking the most recent k entries.
4. Log misses in both directions the paper's two pressures predict: complementary entries left out because their value was under-estimated, and redundant recent entries crowding the window.

If the right memories start arriving at decision time, you have reproduced the paper's central claim in miniature. If they don't, the weak link is usually the value estimate — my own reading of where this rule strains, since the abstract names learned geometry, repeated retrieval, and the full proposed method as what still needs testing.

Connection to Victoria's stated memory design

The model card for Victoria, SentX SI's assistant, describes dynamic symbolic memory in which important memories can strengthen with use and lower-value details can fade, with selective recall and no guarantee of exact retention. The paper's cost-regularised objective is one concrete architectural pattern consistent with that stated behaviour: accessibility weighted by value doing the work that "strengthening" and "fading" describe. The privacy policy states that the specific internal format and storage architecture are not publicly disclosed, so nothing in the public sources establishes that Victoria implements value-indexed traversal. The connection is methodological — a pattern the product's stated design points toward — not an implementation claim.

Source: Victoria AI Model Card; Privacy Policy | SentX SI

For anyone running long agents, the actionable takeaway is the re-framing: the question is not where memory lives or how fresh it is, but what each memory costs to reach and what it adds once reached. One fixed-geometry toy result supports the direction; the rest is a hypothesis worth testing on your own stack before you bet production behaviour on it.