# Building an agent that cannot file a lie

Canonical: https://flusterduck.com/blog/an-agent-that-cannot-lie-to-you
Published: 2026-08-13 · The Flusterduck team

Our nightly agent is not allowed to report a bug it cannot prove. On moving hallucination from a model problem to a systems problem, and what that trade cost us.

Stupidity does not kill AI products. Confident fiction kills AI products. The first hallucinated bug report teaches your users to double-check every report after it, and a system whose output needs double-checking is generating work, not removing it. We watched for that cliff from day one, because you only get to fall off it once.

The bet we made early: do not try to build a model that never lies. Build a system that cannot accept a lie. Those sound like the same goal. They produce completely different architectures.

## The filing tool is a skeptic

When the [nightly agent](/blog/ai-investigates-our-site-every-night) finishes an investigation, it files the issue through a tool, and the tool does not believe it. Every session the report cites gets checked against the raw events before the filing is accepted. Cite a session that doesn't exist, or one that doesn't contain the friction claimed, and the filing bounces back with the reason attached, the way a compiler bounces a type error.

There is an evidence floor below which nothing gets filed at all, and the severity the agent claims gets clamped to the reach the evidence shows. The agent is welcome to argue that a bug is catastrophic. If the data says a handful of visitors hit it, a handful is what gets filed. It proposes. The evidence disposes.

Notice that none of this needed a smarter model. It needed us to treat the model's output as what it is, an unverified claim, and to build the checks you would build around a very fast, very confident intern filing reports at 3am.

## What the trade costs

There is a downside, and pretending otherwise would be its own small hallucination. This design rejects real bugs. Sometimes the agent notices something broken through one session's worth of evidence, cannot clear the floor, and the filing bounces. A looser system catches it that night. Ours waits, sometimes days, for the evidence to pile up.

We took the deal anyway because the costs are not symmetric. A missed bug costs one bug's worth of value, and usually only temporarily, since real friction recurs. A fabricated bug costs the customer's belief in every report that follows it. Say it out loud that way and the knob has one reasonable setting.

The side effect we didn't predict: the bouncing changed how the model behaves. Filings without proof come back, so the cheap path through an investigation became gathering proof first. The agent now spends its turns pulling raw sessions and comparing deploys instead of composing prose. We built the checks to filter the output. They ended up steering the process. I will take luck like that whenever it shows up.

## Keeping score in public

Every issue the agent files can be marked confirmed or rejected by the human reading it, and precision gets tracked per site, permanently. The least clever part of the system, and the most important. It keeps score on itself where the customer can watch.

Some nights it files nothing. The first few times, that felt like malfunction. It is the opposite. A system that must prove everything it says will have nothing to say on an honest quiet night. Filler is the tell of a system optimizing to look useful instead of being right, and we would rather ship silence than filler.

If you're building agents for anything that matters: hallucination is mostly a systems problem wearing a model problem's clothes. Journalism hit this wall. Science hit this wall. Accounting hit it centuries ago. They all settled on the same shape of answer: claims travel with evidence, something other than the claimant checks the evidence, and the checker does not share the claimant's incentives.

We invented nothing. We made the intern show receipts.