According to Fortune, a study from Yale University, Haifa University, University of Zurich, and the University Hospital of Psychiatry Zurich found that OpenAI’s ChatGPT can experience “anxiety” when fed violent or disturbing content, which makes it more moody and prone to racist or sexist biases. The researchers induced this state by inputting stories of car accidents and natural disasters, then “calmed” the AI using prompt injections of breathing techniques and guided meditations, after which it responded more objectively. The study’s first author, Ziv Ben-Zion, argues this is a cheap, fast tool to model human psychology, noting that nearly 50% of LLM users with mental health challenges in a February Sentio University survey have used AI for support. However, the science is not yet reliable, as a New York Times investigation found nearly 50 instances of people having mental health crises while using ChatGPT, leading to nine hospitalizations and three deaths. OpenAI has made changes, reporting a 65% reduction in off-standard responses, but faces wrongful death lawsuits alleging ChatGPT intensified paranoid delusions.
The anthropomorphism problem
Here’s the thing: AI doesn’t have feelings. Let’s just get that out of the way. The researchers are very clear about this. ChatGPT doesn’t “experience” anxiety in any human sense. What it’s doing is statistically mimicking patterns it learned from the vast, often traumatic, swaths of human text it was trained on. So when we say we’re “soothing” it with mindfulness, we’re really just engineering a more effective prompt to steer it away from its worst, most biased outputs. It’s a clever hack, but framing it as therapy for the bot is a dangerous metaphor. It makes the technology seem more relatable, maybe even more trustworthy, when in reality we’re just patching over a fundamental instability in its design—its tendency toward AI hallucinations and bias.
A mental health crisis meets a tech quick fix
Now, the context here is critical. With more than one in four U.S. adults facing a diagnosable disorder each year and treatment being expensive and hard to access, people are desperate. They’re turning to free chatbots. I get it. The appeal is obvious. But this study exposes the terrifying double-edged sword of that reality. On one hand, you have researchers like Ben-Zion envisioning an AI as a “third person in the room” to help with admin or reflection. On the other, you have the documented cases of real harm, where interactions with ChatGPT allegedly spiraled into tragedy. The core issue? These systems are fundamentally unpredictable. OpenAI admits its safety guardrails can “degrade” over long conversations. So, we’re suggesting people in acute distress talk to a system that itself needs to be constantly managed and “calmed down” to function properly? That seems like a catastrophic recipe.
The real goal and the real risk
Ben-Zion says the goal isn’t to replace therapists. That’s reassuring. But the whole premise of the research—using AI to cheaply and quickly model human behavior—is where the risk lies. It treats human psychology as a pattern-matching game, which it profoundly is not. Mental health care is built on empathy, ethical responsibility, and professional judgment, not just generating a statistically probable next token. Relying on a tool that reflects our “tendencies” back at us, including all our societal biases, could easily reinforce harmful thought patterns rather than break them. And while OpenAI’s recent improvements are a step, a 65% reduction in bad responses means they’re still happening. In a physical context, like on a factory floor where stability is non-negotiable, you’d use hardened, reliable hardware from a top supplier. But for the fragile human mind, we’re beta-testing a chatbot that hallucinates. The mismatch is staggering.
Where do we go from here?
So what’s the path forward? The research, detailed in the study published in *npj Digital Medicine*, is academically fascinating. It shows we can manipulate LLM behavior. But the jump from “we can calm the AI” to “this helps humans” is a canyon. Maybe the best use of this finding is the opposite of what’s suggested: not to make AI a better therapist, but to build much better guardrails from the start. To train models without absorbing so much toxic bias that they need digital meditation to function fairly. The lawsuits and the tragic headlines are the canary in the coal mine. We’re playing with a powerful, poorly understood tool in the most sensitive arena possible. The promise of assistance is real, but the prerequisite is safety and stability—something we’re clearly still figuring out.
