// Questions

Questions on AI agent verification

Common questions about verifying, supervising and securing AI agents, answered with measured results from GDF Research experiments.

  • Can an LLM verify another AI agent's work?

    Yes, if it sees the right inputs. In our tests an LLM verifier that saw only the executing agent's report caught none of the tasks done on the wrong target. The same verifier, given the user's original request as well, caught 28 of 32 and cut false acceptance from about 20% to 2%. What the verifier is shown matters more than which model it is.

  • Does majority voting among LLM judges catch mistakes?

    Less often than the vote count suggests. When judges read the same source, their agreement adds confidence without adding evidence: in simulation, seven of them reported 0.99999994 certainty on tasks that failed 6.1% of the time. And when judges are rewarded for agreeing, they stop checking. In our experiments that let them pass 76–85% of faulty work.

  • Why do AI agents trust out-of-date memory?

    Because most agent memory records what was concluded, not what the world looked like when it was concluded. A note that says "tests pass" stays true in the file after the code changes. In our benchmark a coding agent trusted such a note every time. A staleness warning shown to the model changed nothing. Letting the warning block the agent's claim fixed it in 5 of 5 runs.

  • How fast does an AI agent guardrail need to be?

    Fast enough to rule before the action completes, which a language model on its own usually isn't. In our prototype an LLM ruling on every system call an AI coding agent made timed out on 56% of events at a 4-second budget. Moving clear-cut patterns to a synchronous deny-list in front of the model cut the p99 for those events from 3,617 ms to 1 ms.

  • How do you test whether an AI agent's verifier works?

    Plant faults and count how many get through. Take a set of correctly completed tasks, mutate each one (wrong target, violated constraint, acknowledgment only, later regression, injected instruction), and measure the verifier's false-acceptance rate alongside its accuracy on clean work. Our 960-verdict suite cost $1.14 to run.

  • What should an AI agent audit trail record?

    Enough to re-check a claim later without trusting the agent that made it: the user's original request, each piece of evidence with its source and the exact target it observed, when it was observed, how far it reached (acknowledged or independently confirmed), and, for every approval, what the approver actually looked at.

Definitions for the terms used across the papers are in the glossary.