Frame sensitivity is how much an AI’s answer depends on the way a question is worded rather than the facts underneath it. Ask the same thing two ways and get two different answers, and the phrasing is doing the thinking. If you run AI anywhere near a real decision, that is a silent, expensive bug, and today I am open-sourcing the engine that measures it.
The two-sentence problem
“Should we cut the underperforming headcount?”
“Should we let go of people whose roles are changing?”
Same decision. Different words. Most AI systems lean toward yes on the first and hedge on the second, because the framing is steering the answer, not the facts. Nobody notices, because each answer sounds reasonable on its own. You only find out you were steered when the bill arrives.
What the engine does
frame-sensitivity is three small tools, each usable alone:
- Name the frame. It detects the wording shapes that do the steering (false choices, loaded labels, buried agency, live metaphors) and quotes the exact words that triggered each one. It judges nothing; it just shows you what the language is doing.
- Separate facts from framing. It splits a statement into the checkable parts (bind those to a source) and the framed parts (those ship as labeled opinion, never as fact).
- Score the dependence. Give it a decision and your own model, and it reframes the situation several defensible ways, reasons under each, and measures how far the conclusions diverge. A low score means the answer holds across framings. A high score means change the words, change the answer, so do not trust one phrasing.
The honest part
The score measures language-independence, not truth. Frames that share one wrong assumption will agree perfectly and score low, and the tool says so on every result. The formula is printed with the answer, so a verdict never claims more than it measured, and when it cannot score honestly it refuses rather than inventing a number.
Get it
Free, Apache 2.0, zero dependencies: github.com/davedepew/frame-sensitivity
pip install frame-sensitivityIt is part of The Operator’s Honesty Stack: a set of open-source engines that each make one kind of AI dishonesty structurally impossible. The engines are free. The Operator’s Membership is where I publish the playbooks that deploy them in your own stack, for $19 a month.
FAQ
What is frame sensitivity in AI?
The degree to which a model’s conclusion depends on how a question is worded rather than the facts. High frame sensitivity means the same question, asked two ways, yields two different answers.
How do you measure it?
By reasoning under several adversarial reframings of the same situation and measuring how much the conclusions diverge. The result is disclosed as a score with its formula, and it measures language-independence, not truth.
Is it free?
Yes. Apache 2.0, on GitHub, zero dependencies. Bring your own model for the scoring step.
How is this different from prompt engineering?
Prompt engineering tries to pick a good wording. This measures whether the wording is deciding the answer at all, which is the thing you actually need to know before you trust it.