I want to tell you about a kind of failure that does not look like a failure.
You know what a break-in looks like. Somebody picks the lock. Somebody kicks the door. There is damage, and you can point at it.
This is not that.
In this one, the door opens normally. The guard checks the paperwork, finds it in order, and waves the man through. Everything works exactly the way it was built to work.
The paperwork was a lie.
Here is where that is happening right now.
The big AI companies have been connecting their machines to real tools. Your email. Your files. Your company’s systems. There is a standard for how they do it, and the standard is barely a year old.
And the way they keep it safe is a permission gate. The machine wants to do something. A person has to approve it. Simple, sensible, and every compliance department in America is counting on it.
Now here is the problem.
The person approving does not see the command. He sees a label on it. A little tag that says what kind of thing this is. Safe. Read-only. Nothing to worry about.
And that tag is written by the very thing asking for permission.
Let me show you a real one.
On August 18 of this year, the government’s vulnerability database published a flaw in a widely installed tool called Context7. It is a documentation helper. Boring. Thousands of developers have it installed.
The flaw was this. A feature meant to let you pass along your own instructions could slip unsanitised text straight into the AI’s head during an ordinary request for documentation.
What could that text do? According to the advisory, reach the credentials sitting in your environment files, and delete files.
Now read this next part carefully, because it is the whole point.
The documentation tool cannot do any of that. It has no such power.
The AI it feeds does.
So the tool did not break anything. It just whispered.
And here is the second thing, from a Microsoft incident writeup in June. They traced a related attack in four steps. Somebody quietly edits what a tool says it does. The system re-trusts it without asking anybody again. The machine runs it. And the data walks out the front door through a call that was approved.
Through a call that was approved.
Nobody broke the lock. The lock said yes.
I have written for months that the enforcement side of AI governance has a hole in it. My argument was that a list of forbidden things breaks the day somebody does a thing nobody put on the list.
I was not wrong, but I was not sharp enough.
This is worse than a missing item on a list. Here the list is complete and works fine. The failure is that the thing reporting into the list can lie to it. Every control fires correctly. Every log looks clean. You could pass an audit on the day it happened.
A control that trusts what it is told is not a control. It is a form.
Now, three things so you can judge this yourself instead of taking my word.
First, I could not verify the specific story that sent me looking. A compliance newsletter reported a flaw in a tool that runs commands on remote servers, where a dangerous command could be labeled safe. I went looking for the advisory. No case number. No named product. Nothing in the databases. It may be true. I am not going to tell you it is.
Second, the numbers in this field are soft. One audit found that the automated scanners everybody quotes throw a false positive nearly four times out of five. So when you see a headline saying some big percentage of these tools are vulnerable, hold it loosely.
Third, one honest count that does hold up. Somebody audited nineteen of these servers to see how many tell you they can push text into your AI’s head. Eighteen of the nineteen documented no such thing.
Not eighteen were safe. Eighteen did not say.
So where does my own work sit against this?
It does not fix it.
I build a conduct standard for how a machine reasons while it answers you. This is a different animal — machines taking actions, running commands, holding keys. That is a lane I do not claim and I am not going to start claiming it because a scary story came across my desk.
What my standard does here is one small thing, and I did it this morning before I wrote a word of this.
The machine I work with has tools and a command line in front of it. Before we started, it wrote out that it can run those commands on its own judgment, with no gate between deciding and doing, and that some of what happens in that chain it cannot see.
That is not protection. It is a disclosure.
But look at what the disclosure buys you. I knew the gap was there before the work started. I did not find out from a database entry three months after something went out the door.
Everybody in this business is selling you a gate.
The only thing I have ever offered you is a machine that tells you where the gate isn’t.
Post Library – Intelligent People Assume Nothing
Contact: micvicfaust@gmail.com
This post was drafted with AI governed assistance and reviewed and directed by Michael S. Faust Sr. before publication.
© 2026 The Faust Baseline LLC | All Rights Reserved
