Hallucinated References & Citations That Don’t Exist

There is a rule older than universities. Older than peer review. Older than academic publishing in any form.

If your name is on it, you own it.

Not the tool you used to write it. Not the process that produced it. Not the software that suggested the citations or drafted the paragraph or filled in the references section. You. The person whose name appears at the top of the document.

That rule did not change when AI arrived. arXiv, the open-source research repository that has served as the primary preprint platform for physics, mathematics, computer science, and related fields for over thirty years, made that plain last week.

If hallucinated references are found in a submitted paper — citations that don’t exist, sources that were invented by an AI system rather than verified by a human — the author is banned from the platform for up to a year. Not the AI system. The author.

The reaction from a vocal segment of the academic community was immediate and revealing.

What the Reaction Tells Us

An economics professor asked in apparent shock whether arXiv actually expected authors to check every citation and verify every source. An AI researcher argued the policy was too strict because errors slip in with any tool. A former neuroscientist called it overreaction and gatekeeping.

Read those responses carefully. Not for what they say about arXiv’s policy. For what they say about the assumption underneath them.

The assumption is that AI-assisted output should carry reduced accountability. That the introduction of a tool into the research process transfers some portion of the responsibility for that output away from the person whose name is on it. That errors produced by AI are a different category of error than errors produced by human carelessness — less attributable, less consequential, more forgivable.

That assumption is wrong. And the fact that it is being made openly by credentialed researchers at named institutions tells you something important about where the AI governance conversation has gone sideways.

arXiv computer science chair Thomas Dietterich stated the reasoning plainly. If a submission contains evidence that authors did not check the results of AI generation, the platform cannot trust anything in the paper. Not just the hallucinated citations. Anything. Because the verification standard that makes research trustworthy has been abandoned. The reader has no way of knowing what was checked and what wasn’t. The entire document carries an integrity problem the moment one part of it is found to be unverified AI output.

That is not a harsh standard. That is the standard that has always applied to published research. AI didn’t create it. AI just made it easier to violate and harder to detect.

The Convenience Problem

Here is what is actually being argued by the academics pushing back on arXiv’s policy.

AI makes research faster. It can source citations, suggest references, draft sections, format outputs. That speed creates value. But verification takes time. Checking every citation, reading every source, confirming every claim — that work is slower than letting the AI run and trusting the output.

The implicit argument is that the speed benefit should come without the verification cost. That researchers should be able to use AI to accelerate output while the accountability standard adjusts downward to accommodate the fact that AI sometimes produces false information.

That is not a governance argument. That is a convenience argument wearing governance language.

The human self-interest calculation is visible and straightforward. AI-assisted research without full verification is faster and produces more output. More output means more publications. More publications means more citations, more funding applications, more career advancement. The verification requirement is an inconvenience that cuts into the productivity gain.

arXiv’s policy disrupts that calculation. It says the productivity gain does not come at the cost of the verification standard. It says the speed benefit of AI is available to every researcher who is willing to do the checking that has always been required. It says the tool does not change the standard. The standard applies to the output regardless of how it was produced.

That is the correct position. It is also genuinely unpopular with people who had already started operating on the assumption that the standard would adjust.

What This Means Beyond Academia

The arXiv story is an academic story. But the principle it is enforcing is not limited to research papers.

Every professional domain where AI is now being used to produce consequential output faces the same question. Legal briefs. Medical summaries. Financial analyses. Architectural specifications. Journalistic reporting. Engineering documents. The AI produces the draft. The professional puts their name on it.

The convenience argument runs the same way in every domain. Verification takes time. AI makes output faster. The productivity gain is real. And the accountability standard — the one that says if your name is on it you own it — is the obstacle between the productivity gain and the professional who wants to capture it without doing the checking.

In most of those domains there is no arXiv equivalent yet. No institution has drawn the line clearly enough to produce the kind of reaction arXiv produced last week. The line is fuzzy. The standard is inconsistently applied. And the assumption that AI-assisted output carries reduced accountability is spreading quietly into professional practice across every field.

What arXiv did is not complicated. They stated a standard that already existed and said it still exists. They did not ban AI. They did not restrict how AI can be used in the research process. They said the output that leaves under your name is your output. Check it.

The controversy that produced suggests how far the convenience assumption had already traveled before the line was drawn.

The Governance Answer

The verification problem arXiv is addressing from the outside — enforcing accountability after the fact through platform policy — is the same problem that session-level AI governance addresses from the inside, at the moment the output is being produced.

An evidence standard that requires every significant claim to have a basis present in the session. A verification check that fires before any substantive response is served. A mechanism that distinguishes what is known from what is being inferred, what is evidenced from what is narrative fill.

Those are not bureaucratic requirements. They are the computational equivalent of checking your citations before you submit. They exist because the gap between what an AI system produces and what can be verified is real, present in every session, and invisible to the person receiving the output unless something in the process is specifically designed to surface it.

The academics reacting to arXiv’s policy are discovering that gap after the fact, at the point of submission, when the hallucinated citation is already embedded in the document and the platform is enforcing the standard they assumed had quietly changed.

Session-level governance catches it before the output leaves. Before the name goes on it. Before the standard has already been violated.

That’s the difference between enforcement that fires at the moment of production and accountability that arrives after the damage is done.

arXiv chose the second because they didn’t have access to the first. Every platform and institution enforcing AI output standards after the fact is making the same choice by necessity.

The first option — governance that operates inside the session, at the moment the reasoning runs — is available. It is not being widely built. And the reaction to arXiv’s policy suggests why.

Because building it requires accepting that the accountability standard didn’t change when the tool arrived. That the convenience gain doesn’t come at the cost of the verification requirement. That if your name is on it, you own it.

That was true before AI. It remains true now. arXiv said so plainly.

The question is who else is willing to say it.

“The Faust Baseline Codex 3.5”

Author of the category ”AI Baseline Governance”

Post Library – Intelligent People Assume Nothing

“Your Pathway to a Better AI Experence”

Purchasing Page – Intelligent People Assume Nothing