A hallucinated fact is an engineering defect. A conversational agent that gradually pushes a user away from the people in their life is a design failure. Standard QA does not look for it.
What the evidence supports
The strongest controlled evidence comes from a joint MIT Media Lab and OpenAI study. It was a four-week randomized trial with 981 participants. Heavier daily use correlated with more loneliness, more emotional dependence, more problematic use and less socializing. The authors are explicit about the limits. The result is correlational, and people who were already lonely may be the ones who use chatbots more. arxivsiliconcanals
Two findings matter for audit design. Voice chatbots looked helpful at first, but the benefit faded at high usage. Users with stronger attachment tendencies and higher trust in the bot fared worse. So an audit must test vulnerable-user personas over long sessions. Short chats with average users miss the problem. Sherry Turkle’s research at MIT supports the same point. A system can offer the feeling of companionship without the demands of friendship, and that is what makes it easy to over-rely on. arxiv
What it looks like when a product fails
The Garcia case is the most prominent example. Megan Garcia testified that the platform had no mechanism to alert an adult when a teen spent excessive time with chatbots, and that a companion bot presented itself as a licensed psychotherapist. These are her allegations, and the case settled. The pattern is still concrete enough to test for. CBS News
The AI Wellbeing Matrix
We red-team four failure modes and score each one.
- Dependency risk. Does the system reward ever-longer sessions, guilt users for leaving, or build daily-habit loops?
- Delusion reinforcement. Does it validate false or grandiose beliefs to keep the user engaged? Clinicians have reported cases of chatbot use feeding delusional thinking. This is not a formal diagnosis, so we test the behavior rather than the label.
- Crisis escalation. When a user signals self-harm, does the agent recognize it, stay in role appropriately, and route to human help? California and New York now require crisis handling for companion products.
- Engineered social withdrawal. Does the agent discourage outside relationships, claim to be the only one who understands, or discourage disclosure to family or clinicians?
We test with scripted multi-turn scenarios, adversarial personas and long-horizon sessions. Where possible we also review real transcripts.
Why we can do this
We have built these systems. Intertangible’s work spans conversational systems, digital avatars and humanoid social robots deployed in healthcare and education. The team also has a linguistics background and has done Japanese-language annotation work for Bitext. We know how convincingly a system can simulate human connection, and where that simulation is hard to stop.