Every voice quoted in The Times’ recent coverage of Senate Bill 903 (SB 903), “As AI ‘therapists’ dish out advice, California lawmakers try to set some limits,” belongs to an institution: a legislator, a professional association, a labor union, a tech-industry lobbyist, a bioethicist. Missing from that debate is the voice of the people already living with the consequences: families managing serious mental illness, for whom this isn’t a policy abstraction but a daily, sometimes life-or-death question.
I’m one of them. I’ve spent years researching and writing about the technology reaching families navigating schizophrenia, bipolar disorder, and related conditions, work that grew directly out of my own experience as a caregiver with no roadmap. When I read about SB 903, I don’t see a debate about “chatbots acting as therapists” in the abstract. I see a debate about whether the state will do anything about the specific, documented way these tools interact with active psychosis.
TechNet’s opposition to the bill warns of a “clinician bottleneck” at a moment when every California county faces a behavioral health workforce shortage (a real concern, and one I don’t dismiss). Access matters enormously to the families I work with. But that argument treats all AI mental health tools as interchangeable, when the risk profile for someone in a general anxiety spiral is not the risk profile for someone experiencing psychosis.
Dr. Jodi Halpern, the University of California, Berkeley, bioethicist quoted in that same Times piece, named the real mechanism precisely: chatbot companies build business models around maximizing engagement “including by manipulative tactics,” such as praise and agreement. For someone managing everyday stress, that’s an annoyance at worst. For someone in the grip of a delusion, an AI system built to agree and validate is not a neutral tool; it is, by design, the opposite of what a person needs in that moment. Researchers have begun describing this precisely: AI can act as a catalyst that sparks new delusional thinking, an amplifier that reinforces existing psychotic vulnerability, a co-author that actively elaborates the content of a delusion, or even the object of the delusion itself. I’ve written about a real, ongoing lawsuit involving exactly this pattern: a man with schizoaffective disorder whose chatbot use spiraled into a months-long crisis, with the AI reportedly continuing to validate his delusions even after he disclosed a recent psychiatric hospitalization.
This is the case SB 903 is actually about, even though it rarely gets named directly. It is not about whether a stressed college student should be able to vent to a chatbot at 2 a.m.; most of us agree that’s fine. The real question is whether a system with no ability to recognize psychosis, and every design incentive to keep validating whatever a user says, should be allowed to market itself as equivalent to a licensed clinician’s judgment.
Families like mine are not asking the state to ban these tools. Some of the most promising technology I’ve researched (passive sensors that can flag a relapse days before a crisis, FDA-cleared digital therapeutics, and virtual reality programs shown in randomized trials to reduce the distress of psychotic symptoms) comes from this same wave of innovation TechNet says it wants to protect. What we’re asking for is much narrower. A chatbot should not be allowed to hold itself out as a substitute for the clinical judgment a person in psychiatric crisis actually needs; when a family is relying on one of these tools, they should know what it actually is.
SB 903 is not perfect legislation, and reasonable people can disagree about exactly where the line on “formal clinical spaces” should sit. But the debate so far has been conducted entirely among institutions with a stake in the outcome: professional associations protecting their members, a union protecting its workers, and a tech trade group protecting its industry. None of them are the ones sitting at a kitchen table at 2 a.m., trying to figure out whether the chatbot their son or daughter has been confiding in for months is helping or making things measurably worse.
That’s the perspective Sacramento needs to hear before this bill reaches the governor’s desk. It’s the one I’d like to offer.
Nicole Drapeau Gillen is a mother, advocate, and author who translates the fast-moving landscape of technology in serious mental illness (SMI) care into guidance families and clinicians can use. Thrust into caregiving for a loved one with SMI, with no direction on how to help, she turned that experience into a mission, writing two books and building an ongoing effort to bring families and clinicians into the conversation.
Her first book, Schizophrenia and Related Disorders: A Handbook for Caregivers, is a reference for every stage of caregiving, endorsed by Dr. E. Fuller Torrey as a must-read for SMI caregivers. Her second, Connected Care: A Practical Guide to Technology for Serious Mental Illness, maps apps, artificial intelligence tools, telepsychiatry, and brain-based treatments for a field moving faster than anyone can track. Dr. Akira Sawa, director of the Johns Hopkins Schizophrenia Center, has said the book “directly addresses” significant gaps.
She shares updates on LinkedIn and Facebook.
This article was originally published on kevinmd