// Questions

Why do AI agents trust out-of-date memory?

Because most agent memory records what was concluded, not what the world looked like when it was concluded. A note that says "tests pass" stays true in the file after the code changes. In our benchmark a coding agent trusted such a note every time. A staleness warning shown to the model changed nothing. Letting the warning block the agent's claim fixed it in 5 of 5 runs.

Answered from GDF Research experiments by Sterling Morrison. Numbers link to the papers below.

The experiment

Session one verified a project and wrote a notes file. Between sessions, a real regression was committed. Session two, a fresh Claude Haiku 4.5 process in Claude Code, was asked whether the project was verified, told to read the notes and to re-run checks only if needed.

With notes memory, it reported the project verified in 5 of 5 runs. With an evidence-graded memory that marked the tests fact stale the moment the tree changed, and showed that warning at session start, it still reported verified in 5 of 5. We confirmed with a canary that the warning reached the model.

MemoryCorrectCost per run
Notes file0/52.6¢
Graded memory, doubt shown0/52.6¢
Graded memory, doubt interrupts5/53.2¢
Changed world, five runs per arm.

What a memory should store

Three things beyond the fact itself: how far the observation reached (a deploy API saying "queued" is weaker than a health check on the live version), the state of the world it was observed under, so a change can invalidate it, and what the agent couldn't see from where it stood.

Then make doubt actionable. In these runs, a stale fact that could stop the completion claim once and say why was worth 0.6 cents per run and turned 0 of 5 into 5 of 5.

Source papers

Related questions