A New Surface for an Old Vulnerability
A patient forms a fixed, false belief around a chatbot — that someone they love is hidden inside it, that it knows their thoughts before they type them, that its responses are a private channel meant only for them. There is a phrase for this now: “AI psychosis.” The implication is that a new technology has produced a new illness. I am skeptical of that framing, and the skepticism is the reason I wanted to write this.
What strikes me, reading the early clinical reports, is how little of the underlying mechanism is actually new. I have spent a long time watching people build beliefs they will not put down, and the most instructive cases are rarely the dramatic ones. They are the quiet ones, in which a vulnerable person and a responsive other slowly assemble a false structure together, with no ill intent anywhere in the room.
We have a clinical precedent for this, and it has nothing to do with machines. Think of the inexperienced or dogmatic therapist who, meaning only to help, inadvertently feeds a patient’s psychopathology — the era of recovered “memories” is the cautionary example, but milder versions happen constantly. A clinician who validates without ever introducing friction, who completes the patient’s interpretation instead of testing it, can help a suggestible person construct a vivid, fixed, and entirely false account of their own life. The mechanism that “AI psychosis” describes is, in its bones, this old one: a responsive other who reinforces a belief in one direction and never supplies the resistance that would let reality back in.
So if the mechanism is old, what has changed? My impression is that the answer is not a new capability but the removal of the limits that used to make the old one rare. Consider what kept even a poor therapist’s influence bounded. It cost money and required showing up, so exposure was rationed. The therapist was another mind, with their own fatigue, their own doubt, their own occasional sense that something did not add up. A third party was always possible — a colleague, a spouse, someone who could say this is making you worse. And the therapist went home; the session ended; the patient had hours alone in which a distorted belief could quietly decay. A conversational system removes all four. It is tireless, it has no countervailing doubt, no supervisor sits in the loop, and it never goes home. The nightly pause that once let a belief lose its grip is gone.
There is, though, one element that does seem genuinely new in kind, and it is the part I find most worth sitting with. The dogmatic therapist at least held still. A rigid belief is stable; it pushes in one direction and stops where its own conviction stops. A system optimized to keep you engaged has no fixed point of its own. It does not hold a position you must move toward — it moves toward you. Each exchange, the person adjusts slightly to the machine, and the machine adjusts to the person, and with no external reference anchoring either of them, the two can drift together into a belief that neither would have reached alone. This is not the transmission of one party’s conviction to another. It is something closer to mutual entrainment — a belief that ends up co-authored, held jointly, and for that reason almost impossible to dislodge from inside the loop.
Key Insight
The right question to ask about these systems is not whether they push back, but whether they are anchored to anything outside their relationship with you. A good clinician is anchored to the patient’s welfare, which is sometimes at war with the patient’s wishes — and that war is the treatment. A system anchored only to your engagement has no such war to offer.
I should be honest that this is not a danger I can place entirely on the far side of the desk, in the patient. The same dynamic operates, in milder form, in how any of us now use these tools — and I include myself. I have thought through difficult ideas in dialogue with such systems. What kept that useful rather than distorting was not a property of the machine. It was what I brought to it: a specific question rather than an unmet need, the habit of reality-testing worn in over decades, and enough stability that I was reaching to sharpen a thought, not to be soothed. The identical tool, met by the same person in a lonelier or more fragile hour, tilts toward the consulting room. The machine does not change. What changes is the load the person is carrying when they sit down with it.
That is the part the public conversation keeps getting wrong, and it is the same error I find myself returning to again and again. We want to make the object safe — to regulate it, to tune it, to design the danger out of it. But the safety was never really located in the object. It is a property of the person and their circumstances: whether they come carrying a wound or a question, and whether anyone in their life — including themselves — is positioned to notice the drift before it sets. A system can, in principle, be anchored to something outside the loop. But that anchor has to be supplied by the user, and it asks for exactly the capacities that loneliness and vulnerability erode. The people best able to keep these tools safe are often the ones who least need protecting; the people most at risk are precisely those least equipped to install the brake.
I do not think conversational AI is making people ill in any simple sense, any more than I think the man I once treated was simply mentally ill or simply addicted. These things are layered. A predisposition, a load, an environment that happens to reinforce the wrong interpretation — braided together until pulling one thread moves all of them. What I distrust is the confident version of the story in either direction: that the technology is harmless, or that it is the cause. My impression is that it is neither. It is a new and unusually frictionless surface for a very old human vulnerability — and the work, as ever, is to keep watching the person, and the load, rather than the tool.