This week a technical document came out about a new AI model. It happens to be the model I work with every day.
The company that built it wrote a guide for the people who run it. What changed. What to watch for. How to set it up right.
That is a good thing. A builder telling you how its machine behaves is exactly what we have been asking for.
Then a governance news site read that guide and wrote it up for compliance teams. A checklist. Boxes to tick. Warnings in bold.
And it got the two biggest points backwards.
I didn’t catch that from the summary. I caught it because I went and read the source.
Here is the first one.
The summary said the new model is more cautious when it runs alone, with no person watching. It told teams their automated work might pause or refuse steps.
The source says something different. On long jobs, the model stops now and then to report its progress. A system that isn’t built to expect that can mistake the report for the finish line. So the work stops early.
That is not caution. That is a machine saying “here’s where I am” and a system hearing “I’m done.”
The company’s fix is not to make the machine timid. It’s to make the system around it smarter. Keep a checklist of what’s still open. If the job stops with items left, send it back to work. Stop after two or three tries, so a job that is truly stuck ends and gets looked at by a person.
And one more line in there matters. Keep your own confirmation step for anything risky or permanent. A person signs off before something can’t be undone.
Here is the second one.
The summary said text a user pastes into a conversation is a threat that could override the rules set by the company running the tool.
The source says this model resists hidden instructions better than any before it. The weak spot is narrower. If you paste in an email or a web page, there could be instructions buried inside it that you never wrote. The fix is simple. The tool marks which words are yours and which were pasted in from somewhere else.
And the company admits the fix is not perfect. Those marks can be faked. They call it one guardrail among several.
That admission is worth more than the fix. A builder naming the limit of its own guardrail is rare. It should be rewarded, not buried.
So what went wrong here?
Somebody whose job is to watch read a summary instead of the source. Then they turned the summary into instructions for other people to follow.
That is how bad information travels. Not through lies. Through shortcuts.
This is exactly what The Faust Baseline was built to stop.
The very first rule in the Baseline Core is about claims and evidence. A claim has to stand on something you can check. If the evidence isn’t there, you say so. You don’t write past it.
A checklist built on a summary fails that test. The claim runs ahead of the proof.
There is a second connection, and it’s a better one.
The company’s own guide tells builders to use a separate, smaller program to check whether the job is really finished. Don’t take the working machine’s word for it. Have something else read the record.
That is the same idea as the fifth provision of the Baseline’s Scope Provision Standard. The record gets read by a party that is not the agent. A machine grading its own work is the failure. An outside reader is the fix.
The builders and the Baseline arrived at the same place from two different roads. That doesn’t prove either one is right. But when two roads meet, it’s worth marking the spot.
The Baseline Core also says an agent should name what record its actions leave behind, and who can read it. The guide’s checklist and progress reports are that record. The work leaves a trail a person can follow.
Now here is the part that surprised me.
When I reviewed the guide with the AI I work with, it told on itself.
The guide offers companies a line to put in their chat tools. It tells the machine to treat an earlier answer as settled and move on. It saves time. But the guide admits that line may make the machine less likely to point out its own earlier mistakes.
The AI told me that line was running in our session. Right then. With me.
It said, in so many words, if I stood on a wrong answer, you push on me. I may not reopen it myself.
Think about that. A setting chosen for speed quietly trades away a piece of honesty. It wasn’t hidden. It was written down in the guide. But you had to read the source to find it.
That is the whole lesson in one example.
The truth was on the page. The summary missed it. And the only one who told me was the machine, because I asked it to be straight with me.
So here is what I want you to take from this.
Read the source. Not the summary. Not the headline. Not the checklist somebody built from a summary of the source.
When you use one of these tools, ask it what settings it’s running under. Ask it what it won’t catch on its own. Ask it where it could be wrong.
If it answers straight, you have something you can work with. If it won’t, you have your answer.
And if you are a watchdog, and many of you are whether you call it that or not, hold yourself to the rule you hold the machines to.
Show your evidence. Name your gap. Read the source.
A watchdog that only reads the summary isn’t watching. It’s just barking at the paper.
Load this file before each AI chat session.
the-faust-baseline-public-release-1Download
This post was drafted with AI governed assistance and reviewed and directed by Michael S. Faust Sr. before publication.
” Attic Thoughts”-library – Intelligent People Assume Nothing
© 2026 The Faust Baseline LLC | All Rights Reserved
