My agent keeps getting worse as its memory grows — how do I know which memories to keep, and how do I keep the ones I keep honest?

November 13, 2026 · 6 min read

SentX SI

The short answer: your agent writes to memory freely and reads from it blindly. Nearly every lesson it stores gets treated as fact, but most of them were pulled out of a single episode that only said "this worked" or "this didn't" — nothing more. So when the next task arrives, the agent can't separate a real lesson from a lucky accident, and it has no way to check. Two preprints that landed the same week put names on both halves of that problem, and together they sketch what a hygienic loop looks like.

The failure has a name: memory-induced degradation. It is when an agent fails at a task it would have solved with no memory at all, usually because it followed a lesson that was simply wrong, or misapplied or misread one that was right. The cause is structural. Lessons are distilled from trajectories that were labeled only by whether the task succeeded or failed, then stored on their own, stripped of the trajectories they came from. The agent holds the conclusion without the evidence, so it can't verify anything it remembers.

Source: Don't Just Trust Your Memory: Evidence-Linked Memory Reduces Memory-Induced Degradation in LLM Agents

The read side is the fix that keeps evidence attached. WikiTrace stores each lesson as a concise insight, but links it to two things: a summary of why its source trajectory succeeded or failed, and the raw trajectory itself. When the agent faces a new task it starts from the insights, but the moment one is missing detail, contradicts another, or traces back to a failed episode, it follows the link down to the source to fill the gap or correct the insight before it acts. The authors report that across five agentic benchmarks this beats existing memory approaches and a baseline given the same insights alone, and that it cuts memory-induced degradation. That is an author-reported result, not an independent one, and the paper is still under review, but the mechanism is the point: a memory you cannot trace back to is a memory you cannot trust.

Source: Don't Just Trust Your Memory: Evidence-Linked Memory Reduces Memory-Induced Degradation in LLM Agents

The write side is the harder question, because it asks when a new lesson should be admitted at all. FGSE's answer is that a locally successful episode is not enough, and it represents each candidate lesson as a structured hypothesis: a precondition for when it applies, a behavior change, an expected effect, and a verifier for how you would check it. FGSE never commits the lesson directly. It applies the hypothesis in a temporary state and runs three probes — does it transfer to a new target task, can you falsify it, and does it regress any archived behavior you already depend on — then commits, refines, or rejects it on the measured gain and risk. The authors report, at the abstract level, a 39.8 WebArena success rate, 78.8 average completion on AppWorld, and 0.724 and 0.487 on the tau-Retail and tau-Airline tool-use benchmarks, with falsification failures at 5.0%, regression damage at 1.3%, and harmful committed updates at 3.6%. Those are the paper's own headline figures, worth the same author-reported caveat.

Source: When Should Agents Remember? Falsification-Gated Self-Evolution for LLM Agents

Put the two together and you have a write-time hygiene method you can apply regardless of which framework you run.

1. Attach a source to every stored fact. When the agent distills a lesson from an episode, store the lesson with a summary of why that episode went the way it did and a pointer to the raw log. A fact with no source is a liability, not an asset.
2. Write it as a test, not a note. Give each entry a precondition, an expected effect, and a verifier. If you cannot state when it applies and how you would find out it is wrong, you do not have a lesson; you have an anecdote.
3. Gate the write. Stage new lessons in a temporary state. Run them against a transfer probe, a falsification probe, and a regression check before they touch persistent memory, and commit only on measured gain.
4. Re-verify on read. When the agent pulls a memory to act on and the entry is thin, contradicted, or born of a failure, it descends to the source before acting instead of trusting the summary.

Here is what that looks like on a concrete entry, and this is an illustration, not a measured result. Say your agent is a web form-filler that once got through a vendor's checkout.

The flat, unlinked version is "Click Confirm before submitting payment forms." No source, no scope, no way to know whether it came from a success or a fluke, and no way to tell if the vendor has since changed the flow. It will silently rot.

The evidence-linked version is structured: - Claim: on this vendor's checkout, the Confirm step must complete before Submit or the order fails validation. - Precondition: vendor X checkout, layout as of the date it was captured. - Source: the trajectory that produced it, with a note of whether that run succeeded. - Falsification probe: replay a payment form and confirm the validation error actually appears when Confirm is skipped. - Regression check: confirm adding this rule does not break a flow that already worked without it.

The entry commits only if the probe passes, and the precondition gives you a trigger for when to re-run it. The vendor changes the page, the probe fails, and the lesson gets refined or dropped instead of poisoning the next task.

The diagnostic, for when you suspect memory is the culprit rather than capability: take a small set of tasks your agent used to handle well and run it on them twice, once with its accumulated memory on and once with memory disabled. If the memory-on run does worse, you are not looking at a model problem, you are looking at memory-induced degradation, and the unlinked or stale entries are where you start digging. That single A/B is what turns "it keeps getting worse" into a list of specific entries to inspect.

This is the same discipline pointed at a harder target. SentX's model card is blunt about its own memory: recall is selective, exact retention is not guaranteed, and a remembered detail is not proof that it is true. Its checkpoint builder formats your notes but does not verify them, which is the read-side gap in one sentence, a tidy archive that will not check its own entries for you. The two papers are, in effect, building that missing verification step: one keeps memory linked to its evidence so it can be checked, the other refuses to store anything that has not been checked. The habit the model card asks a human to bring to every cited source, open it, read the passage, confirm it says what you think, is exactly what an agent's memory needs, except the agent's sources are ones it generated itself, so the checking has to live inside the write and read loops rather than with a person.

Source: Victoria AI Model Card: Capabilities, Evidence and Limits

The honest limits: both are preprints, WikiTrace is under review at ACL Rolling Review, and every number above is what the authors report on their own benchmarks, not an independent measurement. The environments are specific, web, app, and tool-use agents with verifiers in the loop, and the methods assume you have something to verify against. But the underlying asymmetry is the part that transfers. The gap these two papers describe — free writes, blind reads — is the one their authors call typical, and the two approaches cover its two sides, which is why they read as a pair.

Source: Don't Just Trust Your Memory: Evidence-Linked Memory Reduces Memory-Induced Degradation in LLM Agents; When Should Agents Remember? Falsification-Gated Self-Evolution for LLM Agents