Capability honesty is making an AI report the lowest proven state instead of the highest hoped-for one: it advertises an ability only after a probe has exercised it and passed, and it calls work “done” only when a verifiable artifact backs the claim. An AI that says “I haven’t proven I can do that yet” is worth more than one that always says yes.
The two failures it stops
First, claiming abilities it doesn’t have: registering a capability does not make it real, so here it stays unproven until a probe passes, and a probe that crashes is never read as a pass. Second, saying done when it isn’t: I built the completion gate the day a trial agent reported a task complete with nothing on disk. A completion claim is downgraded to a proposal unless an artifact backs it.
What it does
- Prove before advertise. Abilities are proven by probes at startup; only proven ones are offered, the rest are named unproven.
- Done needs an artifact. “Done” and “claimed done” are different states, kept different in code.
- Refuse, don’t assume. Asking for an unproven capability is refused, not silently attempted.
The honest part
A crashing probe or a raising check is treated as no proof, never optimistically as a pass. The system reports what it proved, never what it hopes.
Get it
Free, Apache 2.0, zero dependencies: github.com/davedepew/capability-honesty
pip install capability-honestyIt is part of The Operator’s Honesty Stack, a set of open-source engines that each make one kind of AI dishonesty structurally impossible. The engines are free; the Operator’s Membership is where I publish the playbooks that deploy them, for $19 a month.
FAQ
What problem does this solve?
AI that overclaims abilities and marks work done that isn’t.
Is it free?
Yes. Apache 2.0, zero dependencies.