My agent remembers a rare case wrongly — how does its stale memory steer the whole group decision?

October 30, 2026 · 5 min read

SentX SI

Through two gates your pooling pipeline controls, and a third leak that starts before any memory is written. That is the shape of the answer in "Rare, Relevant, and Still Wrong: How Memory Fails LLM Group Decisions," a paper by Anouar Imel, Imad Charradi, Abderrahman Sghir and Khalil El Khammar, submitted to arXiv on October 27, 2026 (arXiv:2610.37624). The tested setting is a pooled-memory path many multi-agent setups assume works: a group's discussion is compressed into a short memory, and a final decision maker answers from that memory. The paper's finding is that both obvious compression strategies fail in predictable places, and the decision maker adds a third failure of its own.

Source: Rare, Relevant, and Still Wrong: How Memory Fails LLM Group Decisions

Start with what the benchmark can isolate. The authors built CrossProfile, a diagnostic suite of 60 templates with automatically verified answer keys, in which the ownership of supporting, decoy and irrelevant facts is crossed while the fact text is held fixed, yielding 480 tasks. Because the words never change and only who holds what does, the design separates the two decisions a real pipeline makes: which lines survive compression, and how the decider weighs the survivors. Results are reported across Qwen2.5-7B, Llama-3.1-8B and two DeepSeek models.

Source: Rare, Relevant, and Still Wrong: How Memory Fails LLM Group Decisions

The first gate is selection, and it fails twice over. On hidden-profile tasks, where the decisive facts are spread across members so no single agent holds the full picture, two heuristics look attractive: keep the rarest claims, or keep whatever is relevant to the decision criteria. The relevance heuristic fails because decoys are on topic. The paper reports that even a perfect relevance filter loses 23 to 31 points to rarest-first selection in the classic hidden-profile configuration. A wrong fact that sounds like the answer passes any topicality test. The rarity heuristic is not the rescue: it is no better than random selection on average, and wins only where the rare facts happen to be the useful ones. The line that ties the two failures together is the practical one. Accuracy follows the balance of evidence the memory keeps, not the policy that kept it. No selection rule substitutes for checking what actually survived.

Source: Rare, Relevant, and Still Wrong: How Memory Fails LLM Group Decisions

The second gate is the decider, and it fails by social proof. The decision model gives extra weight to facts attributed to many agents. Removing the attribution tags, or reassigning them at random with line order fixed, raises accuracy in the classic configuration by 13 to 24 points. Holding line order fixed is the important control: the gain comes from the tags themselves, not from where lines sit. This is where a stale memory does its steering. One agent's distorted recollection of a rare case enters the pool, and if other agents echo it, or if the compression step happens to attach it to several sources, the decider treats a corrupted fact as consensus. The crowd-sized endorsement is not something the fact earned; it is something the pipeline assigned.

Source: Rare, Relevant, and Still Wrong: How Memory Fails LLM Group Decisions

The third leak is upstream. In natural discussions, the evidence is lost before any memory acts: the complete transcript yields lower accuracy than no notes at all. The failure is not only what compression drops; it is what the discussion never cleanly produced in the first place.

Source: Rare, Relevant, and Still Wrong: How Memory Fails LLM Group Decisions

Where this sits relative to the usual suspects: our recent companion piece asked why group decisions stay wrong when one member holds the rare fact nobody else has, a structural problem of information distribution where the fix is getting the fact into the room. The scenario in your question — an agent that remembers a rare case wrongly and hands the pool a distorted version — bridges to that piece, but this paper measures a different stretch: not how faithfully any individual agent recalls, but what survives compression of the group's discussion and how the final decision maker weighs the survivors. If a member's recollection is off, the paper does not tell you whether the distortion survives the pipeline; it tells you that once lines are in the memory, accuracy follows the balance of evidence kept, and the decider gives extra weight to whatever many agents are credited with. Missing information and distorted contribution are different failure modes with different checks, and both sit apart from the trust-calibration problem of weighing one assistant's already-received output, and from memory sycophancy, where a stored mistake gets reinforced instead of corrected.

What to do about it. A pre-pool check in three steps. First, before anything is aggregated, have each agent restate its relevant memory in its own words, with a confidence level attached. Second, audit the restatements against the original source before they enter the shared record; the distortion lives in the gap between memory and source, and only the comparison exposes it. Third, strip attribution tags from the pooled record before the decision maker reads it, or at minimum treat "three agents said this" as metadata about provenance, not evidence about truth. To be clear about status: this is an editorial suggestion built from the paper's measured failure modes, not a protocol the authors tested, and it carries no measured accuracy of its own. It formats the notes; it does not verify them. Verification is the audit step, and that one is yours.

Source: Victoria: public model card

The discipline scales from one agent to a group without changing. SentX's own model card says recall is selective and exact retention is not guaranteed, and that a remembered detail is not proof that it is true. Written for a single assistant, those lines are a reminder to repeat what must stay exact. Pointed at a fleet, they become the pooling rule: assume each agent's retained state has drifted, and build the check around that assumption. If you run the check through Victoria, note that the privacy policy states submissions are not confidential and may enter shared memory, so redact sensitive group material before pasting it in.

Source: Victoria: public model card; Privacy Policy | SentX SI

Scope, honestly stated: everything measured here comes from a synthetic benchmark run on four models — Qwen2.5-7B, Llama-3.1-8B and two DeepSeek models — so it maps onto deployed agent teams as a warning, not a measurement of them. The two numbers worth keeping are the ranges. Relevance filtering can cost 23 to 31 points against rarity-based selection in the classic configuration, and attribution tags alone move accuracy 13 to 24 points. Stop assuming pooling is self-correcting. Your pooled record is only as good as what survives compression, and your decision is only as good as how the decider weighs what survived.

Source: Rare, Relevant, and Still Wrong: How Memory Fails LLM Group Decisions