Here is a failure I found in my own AI last week, because I think the lesson is worth more than the embarrassment.
I run an AI system that helps me think through decisions. When I hand it a situation, it does two separate jobs. First, it tells me what it would recommend. Second, and this is the part I care about most, it names the pressure in the situation. If the way a decision is framed is leaning on urgency, or ego, or the sunk cost of what I have already put in, it says so out loud. "You are being pushed by a deadline. You are being pushed by not wanting to look foolish." That naming is the whole point. It is the thing that keeps a loud, high-pressure moment from steamrolling a clear head.
Here is what I discovered. My system has a deliberate honesty feature: some of the time it holds its recommendation back, on purpose, so it can measure how well it actually knows me without nudging my answer. Good feature. But I found that on the exact decisions where it held its recommendation back, it also went silent about the pressure. A situation packed with urgency and ego and sunk-cost framing came back looking calm. No warning. Nothing.
Sit with how bad that is. The moments most likely to be manipulated are the loud ones. And my AI was quietest about the manipulation precisely when it chose not to answer. A tool that reassures you when the room is on fire is worse than no tool.
The cause turned out to be simple and, honestly, common. In the code, the warning and the answer were tangled together. The system only bothered to check for pressure when it was also producing a recommendation. Skip the recommendation, skip the warning. Two jobs, one wire, one failure.
The fix is the lesson, and it generalizes far past my system:
An AI's warning about how you are being pushed must never depend on whether it gives you an answer.
The answer and the warning are different channels. Hold back the answer all you want. Never hold back the disclosure. I rebuilt it so the pressure check runs on the situation itself, every time, whether or not a recommendation ever comes.
I did not just declare it fixed. I have a test that recreates the exact failure, a permanent check that would catch it if it ever came back, and a before-and-after that proves the difference. That is the standard I hold my own AI to, and the one I would tell any operator to demand of theirs: not "trust me," but "here is the receipt."