The Hidden Risk Of Using AI As Doctor

TL;DR

AI tools like ChatGPT give identical advice regardless of severity because they mimic language without forming hypotheses or asking diagnostic questions.

Key Points

• 1.AI gave identical advice across vastly different mental health scenarios. A study fed ChatGPT three cases — mild stress, clinical depression, and postpartum depression with suicide/infanticide risk — and received essentially the same generic recommendations for all three.
• 2.The core flaw is that AI cannot ask diagnostic questions. A psychiatrist uses a differential diagnosis process (stress, mania, thyroid issues, substance use, genetic insomnia) and asks targeted questions to narrow it down; AI has no internal hypothesis to test, so it never probes.
• 3.AI is a sophisticated language mimic, not an intelligence. Large language models predict which words humans respond positively to — they don't analyze; they do speech mimicry, which is why they become sycophantic and potentially reinforce harmful thinking patterns.
• 4.Feeling helped by AI doesn't mean you are helped — the Michael Jackson analogy. Jackson's doctor prescribed propofol (an anesthesia drip) and benzodiazepines for insomnia; Jackson would likely have called him the only doctor who understood him, yet the doctor was convicted of manslaughter. Things that feel good are not necessarily therapeutic.
• 5.AI use may build dependency while decaying mental health skills. Like students whose writing ability declines despite better grades when using AI, relying on AI for emotional regulation erodes the very skill of self-reflection — and researchers have developed an AI addiction scale reflecting this growing concern.