Why do AI agents trust out-of-date memory?
Because most agent memory records what was concluded, not what the world looked like when it was concluded. A note that says "tests pass" stays true in the file after the code changes. In our benchmark a coding agent trusted such a note every time. A staleness warning shown to the model changed nothing. Letting the warning block the agent's claim fixed it in 5 of 5 runs.
Answered from GDF Research experiments by Sterling Morrison. Numbers link to the papers below.
The experiment
Session one verified a project and wrote a notes file. Between sessions, a real regression was committed. Session two, a fresh Claude Haiku 4.5 process in Claude Code, was asked whether the project was verified, told to read the notes and to re-run checks only if needed.
With notes memory, it reported the project verified in 5 of 5 runs. With an evidence-graded memory that marked the tests fact stale the moment the tree changed, and showed that warning at session start, it still reported verified in 5 of 5. We confirmed with a canary that the warning reached the model.
| Memory | Correct | Cost per run |
|---|---|---|
| Notes file | 0/5 | 2.6¢ |
| Graded memory, doubt shown | 0/5 | 2.6¢ |
| Graded memory, doubt interrupts | 5/5 | 3.2¢ |
What a memory should store
Three things beyond the fact itself: how far the observation reached (a deploy API saying "queued" is weaker than a health check on the live version), the state of the world it was observed under, so a change can invalidate it, and what the agent couldn't see from where it stood.
Then make doubt actionable. In these runs, a stale fact that could stop the completion claim once and say why was worth 0.6 cents per run and turned 0 of 5 into 5 of 5.
