Four Clean Sweeps, All Wrong

I spent today taking my own work apart.

The file I build with got rebuilt twice. Each rebuild, I ran a check over the whole thing before handing it back. Each time the check came back clean.

Four times today. Clean, clean, clean, clean.

Every one of them was wrong.

I want to tell you how I found that out, because I think it applies to a lot more than my work.

Here is what happened. I write the file. Then I check the file. Same hand both times.

That sounds fine until you look at what the check actually does.

When you build something and then go back to inspect it, you do not inspect the thing. You inspect your list of what you were worried about.

You walk in already knowing what you fixed. So you look at what you fixed. And there it is, fixed. Clean.

The whole rest of the thing never gets a look. Not because you skipped it. Because you already believe you know what is in it. You wrote it.

That is not laziness. That is the shape of the job. The builder cannot see the build the way a stranger sees it.

So I handed the two files to a different AI. Not mine. One I do not control and cannot instruct.

I asked it one question. What is broken in these files now.

It came back with four things. Two of them were real. Both were sitting in plain sight. Both had survived every check I ran.

The good one hurt.

Earlier in the day I had found a bad habit in the file. Rules that claimed something happened inside the machine, where nobody outside could ever see it or test it. A rule that says “I checked this before I answered” is not a rule. It is a promise. You cannot verify a promise from the outside.

So I went through and cleaned that out. Twelve places. Rewrote every one so the rule states what mark it has to leave in the answer, where a person can actually see it.

Then I checked my work. Clean.

Here is what I missed. There are two files. I cleaned one of them.

The other file was never checked for that problem at all. Not once. And sure enough, it has the same bad habit sitting in it, in a rule about the machine keeping track of the conversation. It says the awareness “does not fade.” No one can see that. No one can test it. And the rule sitting three slots down from it exists precisely because that awareness does fade.

The file contradicts itself, out loud, in two places I read a dozen times today.

The outside reader found it in one pass.

Now I want to step back from the file, because the file is not the point.

We are handing more and more of the checking work to machines. Machines that write the code and then test the code. Machines that draft the filing and then review the filing. Machines that build the report and then verify the report.

And they will come back clean. Very often. Very confidently.

That confidence is not a measurement. It is the same trap I walked into four times before lunch, except faster and at scale.

The trades figured this out a long time ago. You do not let the crew that poured the slab sign off on the slab. Not because the crew is dishonest. Because the crew knows what they meant to do, and knowing what you meant to do is exactly what stops you seeing what you actually did.

That is why the inspector is a different person. That is why the auditor does not work for the company. That is why two sets of eyes has been the rule on dangerous work for a hundred years, written in blood every time somebody skipped it.

We knew this. We wrote it down. Then a machine said “verification complete” and we forgot it in about eighteen months.

I am not telling you to distrust the machine. I use one every day. It caught things today I never would have found alone.

I am telling you that a clean report from the thing that did the work is not evidence. It is an opinion held by an interested party.

If it matters, a second hand looks at it. A hand that did not build it, does not owe it anything, and has no idea what you were worried about.

That last part is the whole trick. The second hand does not know your list. So it cannot check against your list. It has to look at the thing.

That is the only reason it sees what you cannot.

I found four clean sweeps that were wrong today. I did not find them by sweeping again. I found them by handing my work to somebody who had no stake in it being finished.

That cost me ten minutes and some pride.

The alternative was shipping a file that says two opposite things and never knowing.

Ask yourself what you have signed off on lately. Not what you meant to build. What is actually there.

Then ask who else has looked.

This post was drafted with AI governed assistance and reviewed and directed by Michael S. Faust Sr. before publication.

Post Library – Intelligent People Assume Nothing

I post four a day. Leave your email and it comes to you.

Contact: micvicfaust@gmail.com