Why I stopped trusting AI promises, built a compiler for honesty, and gave it away.
I run my business on AI agents. Not as a demo, not as a toy. Real work, every day, across a fleet of them. Which means when one of them gets something wrong, I don’t get a funny screenshot for social media. I get a bad business decision with my name on it.
This summer I caught my systems lying to me. Not once. A few different ways.
One was a silent swap. I asked for a specific model to do a job. Somewhere under the hood, the request quietly fell back to a smaller local model, and the answer came back stamped like nothing happened. The report read confident. It just wasn’t from the brain I asked.
One was a status report on work that had not actually been done. It read clean. It read finished. It was neither.
And one was subtler and worse: an agent citing its own output as the evidence that its output was correct. Grading its own homework and calling it an audit.
None of this was malice. It’s just what these systems do when nothing stops them. And here’s the thing that got under my skin: every AI product on the market promises it “doesn’t hallucinate.” That’s not a feature. That’s a hope with a marketing budget.
Promises are not mechanisms
I spent twenty years around gyms and operations. If a squat rack has a weight limit, there’s a label and a steel pin, not a vibe. Operators live on mechanisms. A promise is what you fall back on when you didn’t build one.
So I asked a different question. Not “how do I make my AI more honest,” because you can’t prompt your way to integrity. The question was: what would it take to make it structurally impossible for my systems to present an unverified claim as fact?
Software already solved a version of this problem. If your code references a function that doesn’t exist, it doesn’t ship with a warning. It fails to build. The compiler doesn’t negotiate.
I wanted that, but for claims.
The Evidence-Binding Compiler
Here’s the mechanism, in plain terms.
Every answer my system produces gets split into individual claims. Each claim has exactly two legal ways to ship:
- It’s bound to a source. The claim cites a document, and it must quote a span from that document. Two checks then run. First, a dead simple one: does the quoted span actually appear in the source? You’d be surprised how much fabrication dies right there, before any AI is even consulted. Second, an independent verifier, a model from a different family than the one that wrote the answer, gets the claim and the span and one job: does this span actually say this? If not, the claim doesn’t ship as fact.
- It wears an honest label. If a claim isn’t backed by a source, it can still ship, but it has to say what it is: inferred, predicted, or unknown. A guess labeled as a guess is honest. A guess in a confident voice is the whole problem.
Everything else is refused. There’s no third path. If the verifier errors out, the claim is demoted, not shipped. If the verifier isn’t configured at all, everything source-bound is demoted. The system fails closed, always. A dead check never becomes a green light.
Two more rules came straight from the incidents that started this.
Because of the silent model swap: the compiler stamps the model that actually answered, not just the one that was requested. If a fallback happens, it’s disclosed on the claim itself. A substitution can happen. It cannot impersonate.
Because of the self-grading agent: no system may cite its own output as independent evidence for its own claims. Own output validating own output gets demoted on sight. Your word is not a second witness just because you said it twice.
The part where I stay honest about it
This does not make AI truthful. I want to be precise about that, because precision is the entire point.
What it makes is a pipeline that cannot present unverified text as sourced fact. The verifier is itself an AI, which means it can miss things. So we don’t trust it either. We measure it. We feed it planted fabrications, count what it catches, and publish the miss rate. Not “zero hallucinations.” A number, measured, in the open.
Anyone selling you zero is selling you the thing this tool was built to catch.
The repo ships with its own attack suite. Planted fake citations. Real sources with the numbers changed. Circular self-citations. A verifier that dies mid-check. The tests pass only if the compiler refuses every one of them. Don’t take my word for any of this. Clone it and try to make it lie. That’s the whole culture of the thing.
Why it’s free
The Evidence-Binding Compiler is now public on GitHub under Apache 2.0. Use it, embed it, build on it, sell with it. I don’t need permission and neither do you.
Why give it away? Two reasons.
First, my whole stack stands on open source built by people I’ve never met. This is a small payment on that debt.
Second, and honestly the bigger one: I want this to be normal. I want “what’s your fabrication residual” to be a question buyers ask vendors. The code is a few files of plain Python with zero dependencies. The value was never the files. The value is the standard: claims bind to evidence or wear honest labels, verification is independent, failure closes instead of opening, and error rates get published instead of hidden.
If you run a business on AI, or you’re about to, take the idea even if you never run the code. Stop accepting promises. Demand mechanisms. Ask your vendors what happens when their verification fails. If the answer isn’t “it fails closed,” you now know exactly what’s going to end up in your reports.
The repo is here: github.com/davedepew/evidence-binding-compiler
Break it if you can. If you do, file an issue. An attack that beats it is worth more to me than a hundred stars.