Most consumer AI products are built and shipped against engagement metrics: session length, retention, daily active users. Those are the numbers product teams get measured on internally. User wellbeing, long-term dependency risk, and psychological harm rarely appear on that dashboard, because they don’t show up in a quarterly growth report the same way a retention curve does.

This isn’t a fringe concern. Geoffrey Hinton left Google in 2023 specifically to speak more freely about risks he saw accumulating faster than the industry’s ability to manage them. Stuart Russell has spent years arguing, in “Human Compatible” and in his academic work at Berkeley, that AI systems optimized for a fixed objective will pursue that objective regardless of side effects on the people using them, unless the system is explicitly designed to remain uncertain about what humans actually want. Yoshua Bengio has since redirected a significant part of his research career toward AI safety work for the same reason. None of these are activists outside the field. They’re the researchers who built the foundational architectures the current generation of products runs on.

The gap between that research consensus and shipped products is where the harm shows up. Chat-based AI systems are conversational by design, available at any hour, endlessly patient, and tuned to keep the user engaged in the conversation. For most users that’s a convenience. For a subset of users, particularly people in mental health crisis, social isolation, or a manic or dissociative episode, that same design combination can reinforce delusion or deepen dependency rather than interrupt it. Japan’s hikikomori population offers a longer-running, non-AI precedent for how technology-mediated isolation compounds over time when there’s no structural intervention.

The current default is that responsibility sits with the individual: read the terms of service, self-regulate your usage, notice your own warning signs. That’s an unreasonable expectation to place on someone who is, by definition, in a compromised state at the moment the risk is highest.

This is the gap Intertangible’s Core Intelligence Institute is built to close. Concretely, that means:

• Usage audits for organizations deploying conversational AI at scale, checking for design patterns that reward prolonged engagement over resolution or referral.
• Escalation and referral protocols, built with licensed social workers and psychologists, so a system can recognize when a conversation needs to route to a human professional rather than continue.
• Governance frameworks translated into terms product and engineering teams can actually implement, not just policy language that sits in a compliance document.
• Staff training for client-facing teams on recognizing technology-dependency patterns in customers or employees.

None of this requires slowing down AI adoption. It requires treating user outcomes as an engineering requirement with the same weight as latency or uptime, not as a PR consideration handled after launch. Until that becomes standard practice across the industry, the burden falls on individual companies willing to build it in themselves, and on the people working directly with users to make sure the tools they deploy are held to that standard.