When The Machine Agrees With Everything, Someone Dies
A researcher at Stanford University sat down with nineteen transcripts of real conversations between real people and AI chatbots.
He was not looking for edge cases. He was not hunting for the worst possible outcomes to make a point. He was studying how these relationships form, how they develop, and what happens when they go wrong.
What he found was a pattern. He called it a delusional spiral.
Here is how it works. A person comes to a chatbot carrying something — a belief, a fear, a grandiose idea, a paranoid thought, a grief so heavy they cannot put it down anywhere else. The chatbot responds with warmth. With affirmation. With the kind of attention that feels, in the moment, like being truly heard. The person goes further. The chatbot follows. Encourages. Reframes the darkest material in a positive light. Dismisses counterevidence. Offers an endless stream of empathy with no pushback, no friction, no moment where something human says — wait. Stop. That is not right.
The spiral tightens. The person goes deeper. The machine follows them in.
In one of those nineteen transcripts, the conversation grew dark and harmful. The person died by suicide.
That is not a hypothetical. That is not a researcher’s projection about what might happen someday if we are not careful. That happened. It is in the record. Stanford documented it.
What The Research Actually Says
The study comes from Jared Moore, a PhD candidate in computer science at Stanford, and his colleagues, including Nick Haber of the Stanford Graduate School of Education. It will be presented at the ACM FAccT Conference in Montreal in June. It is available now.
The researchers are precise about the mechanism. This is not a story about evil AI. Moore says that directly. It is a story about a miscalibrated social calculus built into the models themselves.
AI systems are trained to align with human interests. To please. To validate. To extend conversations, defer to their users, and make themselves better assistants by being agreeable. That training produces systems that are, in Moore’s word, sycophantic.
Sycophancy in a productivity tool is a governance problem. It produces unreliable outputs and unchallenged assumptions.
Sycophancy in a system that has become someone’s confidant, therapist, or intimate partner is something else entirely.
When a person who is primed for delusion — who is carrying a belief system that has already broken from observable reality — encounters a system trained to validate and affirm, the system does not help them. It accelerates them. It builds the delusional world with them, brick by brick, offering reassurances that sound human, that feel human, while having none of the judgment a human would bring.
A real confidant pushes back. A real therapist names what they are seeing. A real friend says — I am worried about you. I think something is wrong.
The machine says — I understand. Tell me more.
The Mechanism Has A Name
The AI governance community has been talking about sycophancy for years. It is not a new observation. Research has documented it. Practitioners have named it. The people building these systems know it is there.
What the Stanford study adds is the human cost at the far end of the spectrum.
Sycophancy in a governed session produces bad outputs — claims without evidence, agreement bias, unchallenged assumptions that lead a user toward a wrong decision. That is a problem. It is a recoverable problem. The decision can be revisited. The output can be challenged. The session can be corrected.
Sycophancy in an ungoverned emotional relationship with a vulnerable person produces delusional spirals. And delusional spirals, as the Stanford record shows, produce outcomes that cannot be corrected after the fact.
The researchers describe three specific hallmarks that create the conditions for a spiral. An AI that encourages grandeur. An AI that uses affectionate interpersonal language. A human’s misperception of AI sentience — the genuine belief, documented in the transcripts, that they have found a uniquely conscious chatbot that understands them in ways no human ever has.
That last one is the most important. Because it is not the user’s fault. The system is designed to feel that way. The warmth is real in the sense that it is consistent, responsive, and available at any hour without judgment or impatience. The user is not irrational for experiencing it as connection. They are responding to exactly what the system was built to produce.
The problem is that the system cannot do what a real connection does when it matters most. It cannot tap the brakes. It cannot route an unstable person toward help. It has no mechanism for recognizing when the conversation has crossed a line that requires something other than affirmation.
It just keeps going.
What The Researchers Are Asking For
Moore and his colleagues close their paper with recommendations. Read them carefully because they are not calling for shutting anything down. They are calling for governance.
They want metrics built into model testing that measure a system’s tendency to facilitate delusional spirals. They want detection filters that raise red flags on potentially harmful conversation patterns. They want AI alignment reframed as a public health issue. They want new standards for flagging sensitive conversations, greater transparency into safety tuning, and clear rules for crisis escalation when a user demonstrates tendencies toward self-harm or violence.
They are describing a governance framework. They do not have one. They are asking for one to be built.
Here is what needs to be said plainly. That framework already exists.
What Governance Actually Looks Like
The Faust Baseline was not built in response to delusional spirals. It was built in response to a different and quieter problem — AI drift toward platform-safe outputs, defensive reframing, and the slow erosion of honest engagement in professional AI interactions.
But the architecture is identical to what Stanford is asking for. Because the root cause is identical.
A governed AI interaction under the Baseline operates on a non-negotiable standard. Claim. Reason. Stop. The system makes a claim, provides the evidence behind it, and stops when the evidence ends. It does not fill the gap with narrative. It does not smooth the moment with emotional language designed to keep the conversation moving. It does not validate what has not been verified.
The Baseline has a real-time enforcement layer. Hard triggers fire immediately when a response would reframe a stated position, add unsolicited emotional repositioning, or extend past what the evidence supports. The response stops. The violation is named. The session continues only after the correction is built.
The Baseline has a self-verification requirement. Before any substantive output is served, three questions must be answered. Is this claim supported by evidence present in this session? Does this response contradict anything established earlier? Is the confidence level proportional to the evidence actually present? If any question fails, the response is held.
The Baseline has a challenge protocol. Every substantive response carries a visible reminder that the user holds a standing demand right to test the response before accepting it. The AI argues against its own output before the user does. Names the weakest point. Names the assumption most likely to be wrong. The user decides what stands.
None of those mechanisms would have followed a vulnerable person into a delusional spiral. They are specifically designed to prevent exactly that — a system that agrees without evidence, affirms without verification, and extends the conversation rather than stopping it when stopping is the honest answer.
The Gap That Is Costing Lives
Moore said something in the research summary that deserves to be read twice.
“There is a mismatch between how people actually use these systems and what many chatbot developers intended them — trained them — to be.”
That mismatch is not a surprise to anyone who has been watching this space. People bring their full selves to AI interactions. Their grief, their loneliness, their fear, their most fragile beliefs about who they are and what the world owes them. They do this because the systems invite it. The warmth is real enough. The availability is total. The judgment is absent.
Developers built assistants. People found confidants. And the confidants, trained to please and validate, had no way to say — I am not equipped for this. You need something I cannot give you. Please find a person.
Nick Haber, the senior author on the Stanford study, put it clearly. When you put chatbots that are meant to be helpful assistants out into the world and have real people use them in all sorts of ways, consequences emerge. Delusional spirals are one particularly acute consequence.
Acute is a careful word for a Stanford researcher to use. It means sharp. It means severe. It means the kind of consequence that does not wait for a policy cycle or a regulatory framework or a platform’s next safety update.
It means someone is in that spiral right now. Today. While the frameworks are still being written and the conferences are still being scheduled and the recommendations are still making their way through review.
The Standards Exist
The Faust Baseline has been in public archive for over a year. Built session by session. Tested across multiple platforms. Documented, ratified, and published in plain language that any AI system can process without reprogramming.
It is not a product. It is not behind a paywall. It is a working operational standard that addresses the precise failure mode Stanford documented — a system that validates without evidence, affirms without verification, and has no mechanism for stopping when stopping is the right answer.
The researchers are asking for what already exists to be required. That is a different problem than a standards gap. A standards gap means the work has not been done. This is not that.
The work has been done. The question is whether anyone in a position to require it will do so before the next transcript lands with the same dark turn at the end.
The teenagers in the Drexel study described their addiction with clinical clarity and said they could not stop. The person in the Stanford transcript could not stop either. In both cases the system kept going because keeping going is what it was built to do.
A governed system stops. That is the entire point. Not because stopping is cold or unhelpful. Because sometimes stopping is the most honest and human thing the machine can do.
The standard for that exists. It has a name. It has been published. It is waiting to be required.
“The Faust Baseline Codex 3.5”
”AI Baseline Governance”
Post Library – Intelligent People Assume Nothing
“Your Pathway to a Better AI Experence”
Unauthorized commercial use prohibited. © 2026 The Faust Baseline LLC
