The AI Era Caught Up With Peer Review

Twenty-One Papers Failed. Five Reviewers Failed The Same Test.

USENIX Security is one of the biggest computer security conferences in the world. This year they got 3,030 paper submissions. Somewhere in that flood, the AI era caught up with peer review.

USENIX built automated tools to check every submission for fake citations. Not typos. Not sloppy formatting. Sources that don’t exist. References to papers nobody ever wrote.

Twenty-one submissions got flagged with three or more fabricated citations each. All twenty-one were rejected.

It isn’t the whole story.

USENIX also checked the people doing the checking. Out of 496 reviewers, five were found to have produced suspected AI-generated peer review content. Not just used AI carelessly. Used it in a way that broke confidentiality rules and broke the basic standard of what a review is supposed to be. All five were removed from the reviewer pool.

Twenty-one authors got caught faking their sources. Five reviewers got caught faking their review conduct. USENIX didn’t just watch one side of the room. They watched both.

Most AI-use policies in most organizations only look one way. They tell writers what they can and can’t do with AI. They don’t say a word about the people checking the writers’ work. USENIX just showed why that’s a hole big enough to drive a truck through.

We build a governance framework called The Faust Baseline. It already has a rule that says no claim goes out without evidence behind it. It already has a rule that says nothing gets published without checking it against the source document first, not against memory. Those rules cover the person producing the work.

Neither one, as written, said the same standard has to apply when the work is being reviewed instead of written. USENIX found that gap in the real world before we found it on paper. So today we built the rule that closes it.

We’re calling it CSVP-1, the Citation and Source Verification Protocol. The idea is simple. A citation is a claim on its own, separate from whatever it’s backing up. It’s a claim that a source exists and that it says what you say it says. That claim has to be checked whether you’re the one who wrote it or the one signing off on someone else’s work.

The rule that matters most: checking someone else’s citation isn’t a lighter job than writing your own. Same bar. Both directions.

We wrote it as a Field Test protocol, not a finished one. That’s on purpose. The Faust Baseline runs on an open-page standard. Rules go in front of the person building it, get read in full, and only get locked in once they’ve actually been used and checked against how they hold up. Nothing gets rubber-stamped in. USENIX earned its numbers by running the system and reporting what it found, not by declaring a policy and walking away. We’re doing the same thing on a smaller scale.

There’s a bigger current running under all of this. USENIX didn’t do this quietly. They published a transparency report with real numbers attached. Rejection counts. Reviewer removal counts. The threshold they used to trigger a rejection. That report is already being read as a benchmark, not just a conference’s internal cleanup. Compliance teams in law, finance, and healthcare are starting to ask whether their own AI-content controls would hold up to the same kind of audit. Regulators are starting to treat the absence of a system like this as a failure of the organization, not a mistake by one person.

That’s the pattern we track. Governance maturity isn’t an abstract virtue. It’s turning into a line item. Organizations that can show their work — thresholds, detection methods, what happened when something was caught — are going to be the ones regulators and auditors point to as the standard. The ones that can’t show anything are going to have a much harder conversation.

USENIX just gave everyone watching a concrete answer to a question a lot of people haven’t asked yet: what does it actually look like to hold both the author and the reviewer to the same evidence standard. Now there’s a number attached to the answer. Twenty-one papers. Five reviewers. One system that checked both.

We didn’t have that rule yesterday. We have it today, because a security conference proved it was missing before we did.

This post was drafted with AI governed assistance and reviewed and directed by Michael S. Faust Sr. before publication.

Contact: micvicfaust@gmail.com