Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- What the everliving..... There is no world in which current LLMs are even non harmful for people with actual mental issues.
People making such decisions should be barred from any kind of social adjacent decision-making.
- (Heck, if I may: they have, at best, not been disproven to be non-harmful for people without mental issues, while - it seems, more and more - proving quite harmful to those "borderline" or with underlying, unsurfaced issues.-)by Bluestein
- It's an amazing and disingenuous posit that the problem here lies with the Algorithm as opposed to the consequences under which it was introduced - cost-cutting. The barriers to care for patients who are already struggling with serious mental conditions are not a new phenomenon related to automated tooling derived from best practice corpus. Rather, they are a direct result of a decline in triage staffing.
Take cardiologist Dr. Lee Goldman’s chest pain decision-making algorithm, originally devised at Cook County Hospital to diagnose heart attacks.
Effectively a poster in the ER illustrating a simple decision tree, it outperformed traditional doctor diagnoses; achieving over 95% accuracy and proving 70% better at identifying patients not having a heart attack. This landmark clinical protocol represented a quantum-leap in outcomes, eventually refined to the Revised Cardiac Risk Index (RCRI) used as the gold-standard today.
So not only are algorithmic approaches old-hat in clinical settings, they are in fact a core component of contemporary best practice medicine. But that's somewhat besides the point when much of the issues raised in the article above refer to issues of executive dysfunction - areas in which low-context Agentic AI are basically designed to remedy, e.g.
// Instead of being referred directly by primary care doctors and contacted to schedule appointments, patients were routed through an app, nonclinical call center staff or simply given a phone number to call. “Now the onus is on the patients to be their own care coordinators,” she said... This dynamic can be particularly challenging for some patients. “Lack of motivation and lack of follow-through are the most common symptoms of depression on the planet,”
So the answer here is clearly an increase in appropriate AI resources, rather than throwing the baby out with the bathwater. If we trust a decision-tree poster to do cardiovascular triage better than the instincts of cardiovascular specialists, and OCR models to interpret results better than veteran Radiologists, why not Agentic AI to do the basic work of scheduling appointments?
by piltdownman - RCRI is deterministic calculator, proved improve outcomes, not patient directly use
Is same true for AI triage?
This seems like it's working, actually. Similar to automated Amazon warehouses experiencing a seemingly paradoxical increase in injury rates compared to non-automated ones.....it's very possible this is simply selection bias and the patient triage overall is improved greatly.This change has coincided with a sharp increase in the number of patients who are upset by the time they speak to her. On a typical day, as many as a third of her almost two dozen triage calls are with patients who have struggled to access appropriate care.by scld- This is ignoring that patients were being routed correctly before. This isn’t someone at a help desk. This is a mental health clinician being given the same patients after a worse intake process.by ofjcihen
- Being upset over obtuse process by the time you speek to a doctor is not a sign of good triage. Good triage does not implies angry patients.
Also, amazon injury rates were not paradoxical. If you dont care about people when designing process, it is unsurprising when you end up hurting them.
by watwut - Not sure that assertion holds; looking into the Amazon thing, it looks like the mechanism there was that it automated the "safe" parts of the job (notably walking), and greatly increased the rates at which workers were expected to perform the parts likely to result in injury/RSI, with less downtime between. That's not a flattering comparison to be making for any health system.
- One problem with Kaiser (probably not only Kaiser) when it comes to specialists like mental health workers is that at least in California there was a policy of not allowing specialists to refuse new patients. So over time, specialists' backlogs get bigger and bigger and they are juggling more and more patients and it becomes harder for the specialist to keep track of what's going on. Meanwhile the wait for a new appointment becomes longer as a result - typically months. As a Kaiser patient I personally had multiple specialists quit Kaiser to work elsewhere and at least one cited this as the reason when they told me they were leaving.
It seems really nasty to pair this with reductions in triage staff and the use of algorithmic triage. Their people are already overworked.
by kg - I work in nonprofit mental health and am a Kaiser patient, and I know multiple clinicians who left our org to work at Kaiser. KP has never really cracked the nut of appropriate mental health care. For years they decided it was cheaper to pay the fines for their violations of "Timely Access" laws (requiring that newly referred clients actually schedule/complete first visit within a certain number of days) than to properly staff mental health. I know they also push a lot of people into group mental health encounters, which are unsatisfying and not the right answer if you need to discuss private matters with just a clinician.
I'm not surprised that they're now farming out triage to the robots, or that specialists now have burgeoning wait lists and are leaving KP in frustration. No clear answer other than "try something new besides all the stuff that isn't working".
by SoleilAbsolu - > This dynamic can be particularly challenging for some patients. “Lack of motivation and lack of follow-through are the most common symptoms of depression on the planet,” Marcucci-Morris said. “We wouldn’t tell a paraplegic, ‘Hey, walk down the hall in order to get your wheelchair.’”
This was something I really struggled with getting an ADHD diagnosis as adult. I needed an incredible amount of executive function (dozens of phone calls, navigating the mire of health insurance to figure out who I was allowed to go to, who would take me, being bounced around) to be allowed medication to treat my executive dysfunction. I recall wondering how someone less functional than I was was ever expected to get help.
- Yup. With ADHD you constantly feel behind the 8-ball, forever in a deficit of agency. Worse when it's paired with anxiety so you never feel you are in control or make the right decision. Most of your time is spent worrying about what you didn't do and what you might have missed. Some things have higher priority to worry about so "small" things like calling that ADHD doctor fall by the wayside. Inevitably the chaos catches up where you forget an important event, forget to pay a bill, etc, and you're in panic emergency mode trying to fix it. So of course you forget to call that ADHD specialist. All day every day is like this.by MisterTea
- LLM can't have legitimacy in doing things to human beings, because it's impossible for LLM to have political legitimacy. Political legitimacy in modern societies is already a vicious zero-sum game with a tiny elite having any in the first place, none will be given to objects. Mental health is inextricably linked with violence and political legitimacy therefore, so the triage lacking efficacy or not doesn't even matter - it doesn't have legitimation. The violence inherent in triage is not justified.by curuinor
- It's impossible for corporations to have political legitimacy. Arguably, AI illegitimacy is a subset of that broader umbrella of illegitimacy.by kelseyfrog
- One of the most basic safeguards -- necessary, not sufficient -- for AI and even general technology in healthcare should be transparency. Keep the model a black box (if you really must), but every single input, output, and decision should be logged and available to the patient on demand, no questions asked.
That makes it a lot easier to deal with things like the bad AI transcriptions going around. Get burned by AI that can't hear what you're saying? It's in your file, what it heard and then what it screwed up. Providers don't like everything being on the record like that? Maybe they shouldn't be trusting that tool.
Want to know what the useless AI saw when it sent you to group therapy for acute mania? It's in your file. Providers don't like everything being on the record like that? Maybe they shouldn't be trusting that tool.
- This was the premise of GDPR (which I wholeheartedly support), but Europe caved in to US dictatorship (like Europe always does, cause it has no agency).