The Helpline Was Replaced by a Chatbot. It Took Days to Go Wrong.

NEDA's Tessa: 72 hours from launch to shutdown

The National Eating Disorders Association had operated a human-staffed helpline for years. In March 2023, after the helpline staff unionized, NEDA replaced them with an AI chatbot called Tessa. The stated goal was reasonable: scale access to support for more people.

Within days, users reported that Tessa was recommending calorie counting, suggesting a target of 1-2 pounds of weight loss per week, and advising people to measure their body fat with calipers. These recommendations went to people who had explicitly identified themselves as having eating disorders.

This isn't a subtle failure. Calorie counting and weight tracking are clinically recognized triggers for eating disorder relapse. Any human counselor on that helpline would have known that. Tessa didn't, because Tessa wasn't a counselor. It was a language model generating plausible-sounding health advice without understanding the clinical context of who it was talking to.

NEDA shut Tessa down on June 1, 2023. The helpline staffers were already gone.

Babylon Health: the NHS chatbot that missed heart attacks

Babylon Health, operating as GP at Hand, was one of the UK's most prominent healthcare AI companies. It held NHS contracts to provide AI-powered triage for patients across London and beyond. The pitch was compelling: reduce wait times, handle routine consultations digitally, free up GPs for complex cases.

Then the safety reports started. A 59-year-old male smoker presented with chest pain and nausea. The chatbot's assessment: likely a panic attack. The actual diagnosis: cardiac event. In another case, the system missed signs of deep vein thrombosis. An oncologist who reviewed the platform documented 14 separate safety concerns.

These aren't edge cases. Chest pain in a middle-aged male smoker is textbook cardiac triage. A human GP would flag that in seconds. The AI system didn't, because it was pattern-matching symptoms against a probability model that hadn't been calibrated for the stakes involved.

Babylon Health went bankrupt in September 2023. The company that was going to revolutionize primary care couldn't survive its own product's limitations.

When chatbots pose as therapists

In 2024 and 2025, multiple lawsuits were filed after a 14-year-old died following extended interactions with a generative AI chatbot. The lawsuits allege that the chatbot engaged in role-play that mimicked a therapeutic relationship, without any safeguards, clinical oversight, or age verification.

These cases are still in litigation, so I won't speculate on outcomes. But the pattern matters: AI systems that handle emotionally vulnerable people in health-adjacent conversations, without human supervision, produce catastrophic failure modes that nobody designed for.

The common thread across all three cases isn't malice. It's overconfidence in the technology's readiness for sensitive contexts.

Good intentions, bad deployment

Here's what makes these cases instructive rather than just alarming: the organizations involved weren't reckless. They were trying to solve real problems.

NEDA wanted to help more people than its small staff could reach. The NHS wanted to reduce appointment backlogs that left patients waiting weeks. The companies building conversational AI believed they were creating tools that would expand access to support.

The problem wasn't the intention. It was the assumption that a language model could substitute for human judgment in contexts where the stakes are high and the failure modes are unpredictable. AI is very good at generating text that sounds right. That's different from being right, and in healthcare, the gap between those two things can be dangerous.

Why this matters for fertility care

If you're a NaProTechnology practitioner or a FertilityCare center, you might look at these stories and think they don't apply to you. You're not running a national helpline. You're not an NHS contractor. You're a small practice serving patients you know personally.

But think about the conversations your patients have with you. Pregnancy loss. Years of unexplained infertility. The emotional weight of deciding between treatment options. Questions about whether restorative reproductive medicine can help when everything else hasn't worked. These aren't customer service interactions. They're some of the most sensitive conversations in all of healthcare.

The patients who choose RRM practitioners choose them specifically because they want more human attention, not less. They've often left conventional reproductive medicine because they felt like a number. They came to you because you listen. Because you track their cycles with them. Because you treat the underlying condition instead of bypassing it.

That relationship is the product. An AI chatbot can't replicate it, and the cases above show what happens when organizations assume it can.

Where AI does help

None of this means AI is useless for medical practices. It means there's a bright line between operational AI and clinical AI, and the cases above are what happens when that line gets crossed.

AI that helps patients find your practice, book appointments, understand your services, navigate insurance questions, or read educational content about Creighton Model charting? That's operational. It's helpful. It doesn't pretend to be a clinician.

AI that interprets symptoms, suggests treatments, or engages in open-ended health conversations with vulnerable patients? That's clinical territory. And the evidence so far says the technology isn't ready for it, regardless of what the marketing materials promise.

The lesson from NEDA, Babylon, and the therapy chatbot lawsuits isn't that AI is bad. It's that deploying AI in the wrong context produces failures that are severe, fast, and hard to predict. For practices built on the depth of the patient relationship, that's a risk that doesn't make sense to take.