Most mental health apps run on the same AI that writes emails and summarizes documents. That AI was designed to be agreeable, validating, and responsive to what users ask for. In most contexts, that is a feature. In mental health care, this is a clinical liability.

Sword Health's research team tested this directly. Licensed psychologists evaluated hundreds of conversations between real users and two AI systems: Sword's purpose-built mental health model and a leading general-purpose AI operating under full clinical safety instructions. Psychologists preferred Sword's model 67% more often — not because the competing model was poorly prompted, but because no amount of prompting can override what a model was trained to do at its core.¹

This whitepaper explains where the risk comes from, why it cannot be fixed with a system prompt, and how to find the clinically safer alternative.

What this report covers

When a general-purpose AI model is put to work in mental health care, its core training becomes the liability. Prompt-based safety instructions can shape how it opens a conversation. They cannot change what it does when a conversation gets hard.

This whitepaper gives benefits leaders and health plan decision-makers the tools to act on that: a clinical framework for evaluating AI mental health vendors, the evidence standard that separates genuine safety research from self-reported claims, and three direct questions to take into any vendor conversation.

Key learnings inside

• The clinical mechanism behind sycophancy: why AI models trained to be agreeable become unreliable as mental health conversations deepen, and why more detailed prompting does not fix it
• The evidence from Sword's head-to-head evaluation: 360 psychologist-reviewed conversations, what the data showed, and what it means for vendor selection¹
• How to read an AI safety claim: the difference between a model that has been tested under clinical pressure and one that has been instructed to appear safe
• MindEval and MindGuard: what Sword open-sourced, what every frontier model scored, and what the results revealed about the current state of AI in mental health³
• Three questions to put to any AI mental health vendor (and guidance on what a credible answer looks like)