Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- What can go wrong ? I hope sometimes people would stop and think the implications or at-least debate about it. If someone says “what makes you think it was not debated?” Then I would say it’s definitely not debated enoughby bicepjai
- People will die because of this. Everyone who signed off on this should be in prison.by razorbeamz
- If they think they will save money here on human dispatchers, they could not be more wrong, this will be sued into oblivion after first few deaths caused by AI incompetence to deal with non-standard situations. And I am afraid the worst part is the people who greenflagged this, will walk free.
In case of major event they could just use prerecorded message at the beginning of the call to give people some simple advices for those who can call later, that's what we did in insurance claims department when the line was overloaded. By my experience sometmes dealing with pretty non-standard stuff + now experiences dealing with AI daily for work, this will 100% go wrong despite whatever their press release says about 100% success rate in 3 months (does anyone believe 100% rate?).
by Markoff - I don't know how I'd react if I called 112/911 in an emergency and it was obviously a AI generated voice picking up. Add in the potential for misunderstandings during a situation like this, and my head might just explode on the spot if any misunderstanding happened.
But reading about how it's supposed to work:
> This is intended to handle the increase of calls that are related to one incident, so callers get automatically routed to an AI agent who asks if they are calling regarding the incident. If the answer is yes, then callers can receive information or updates, and if it's no, then the callers are transferred to a human.
Couldn't a human resolve such thing themselves in about 3 seconds? "911, Hello, do you have an emergency?", "No I was wondering abo...", "Ok, redirecting your call now <BEEP>", why do they have to add a slow and infallible AI model here?
- In a situation where you don't have enough people, keeping them on this task takes up someone who could be processing people, people who may be panicking too much to quickly give a yes/no.by TOMDM
- Because a computer can handle 100s of such calls simultaneously.
Similarly, I worked on a system that used AI for approvals of payment for medical activities (in Germany) - it only could fail in favour of the patient (approving when a human would disapprove), thus the ethical issue was fear that reduced workload would cause layoffs for the unionized approvers.
by p_l - > callers get automatically routed to an AI agent who asks if they are calling regarding the incident. If the answer is yes, then callers can receive information or updates, and if it's no, then the callers are transferred to a human.
Why use something as error-prone as an AI for this simple task? I hope they at least add a "Press 1 for Yes, 2 for no" as a backup.
by armada651 - Isn't there like a $50,000 federal fine if the call isn't answered by a human within 5 seconds or did the federal regime suspend that too?by inigyou
- "For instance, it's possible AI could develop hidden biases. If AI is used for predictive policing algorithms, generating an analysis for when and where crime are most likely to occur, it's possible for results to be skewed by historical crime data from over-policed neighborhoods and marginalized communities."
I really don't understand that concern... So you have either a STT-LLM-TTS or fully native speech model that is instructed to answer calls. When does it generate an analysis? Are you giving the agent some tools to generate the analysis ? Maybe don't.
Also, in general I feel like training some classifier would be slightly better from a humanitary point of view, but I agree that in general reducing the work load of overworked public workers should be a priority of AI application.
by Otterly99 - I think the fear is that the people in charge may be too gung-ho about AI and go do something like this, rather than saying that LLMs will spontaneously develop some kind of data analytics power.
- > This is intended to handle the increase of calls that are related to one incident, so callers get automatically routed to an AI agent who asks if they are calling regarding the incident. If the answer is yes, then callers can receive information or updates, and if it's no, then the callers are transferred to a human.
You don’t need ‘AI’ for this really - all it needs is a prerecorded message asking that question and if the caller presses a button it continues. Or if they don’t respond for N seconds it puts them through to an operator. I guess you could use the AI to record that message ahead of time if they really have to, but it doesn’t need to be running in real time responding to people.
We’ve had stuff like this in the UK for years now to do this to prevent accidental dialling (where it will stop to ask “you are calling XYZ service” please confirm, kind of thing)
by dwroberts - The potential I see for AI to help here is identifying and recording a message for "if you're calling about the whatever that just blew up on maple street press 1" situations because obviously the operators will be too bogged down to do that in the moment.
- > all it needs is a prerecorded message asking that question and if the caller presses a button it continues.
This actually seems better with very little overhead. I'd very much prefer hearing a human recording than AI bot.
by wraptile - I don't agree with the luddite-esque AI views like using ChatGPT makes you dumber and it thinks for you, but- this is a real case of why people are angry and why they should be concerned about deployment of AI.
The article says it's basically to handle high call volumes maybe during some kind of widespread incident. But in those cases why would AI even be needed- if calls are flooding in for something and people need to be put on hold why do you need AI to ask them if they are calling about incident x? Just automate a voice for that and ask them that while they're on hold / when the system first picks up.
This smells like a bandaid over an underfunded system, and a way to sneak in further cost-cutting in the future where the real calls will be actually answered by AI.
by awongh - > This smells like a bandaid over an underfunded system, and a way to sneak in further cost-cutting in the future where the real calls will be actually answered by AI.
I don't get why are people so against cost cutting and use of AI in government services?
So much of my tax money is already wasted on bullshit. If you are in Europe - it's majority of your work, being set on fire by bureaucrats.
by dzhiurgis - several things: AI won't make it cheaper, an automated voice system will invariably lead to frustration.
conclusion: have emergency protocols where you can staff your call centers appropriately
by wolfi1 - Having someone answer and provide updates or screen for new information is valuable. For instance someone calling can know exactly where the shooter is. Or it could be something related. Information is valuable. What's the alternative? Not have someone pick up the phones? No matter how much you over staff there will be times where peak volume will be more than you can supportby bko
- > Just automate a voice for that and ask them that while they're on hold / when the system first picks up.
Is this criteria satisfied if we swap the titular “AI” with our inoffensive “automating the voice”?
by refulgentis - What about cases where e.g. there's an earthquake with thousands of incidents and you need to triage help to the worst? It's not common, but there are absolutely cases where small amounts of intelligence applied in parallel is a genuine advantage.
- > This smells like a bandaid over an underfunded system, and a way to sneak in further cost-cutting in the future where the real calls will be actually answered by AI.
I think there’s also an aspect of accountability-washing: when regular 911 makes mistakes, managers and politicians get hauled into the spotlight to explain the failure (my city has that now due to the outgoing mayor’s decision to put an unqualified crony in charge). I will bet that the first time this fails badly it’ll be blame shifting and assurances that some huge tech company has fixed the problem and it’ll never happen again, with as little talk about oversight responsibility as they can get away with.
by acdha - Knowing the way in which bureaucracies work, I suspect there is zero (or likely negative) money available for new humans or old-school systems, but heaps available for anything with AI in the title. A manager who increases human staff will be blamed for causing bloat, but any manager who manages to replace humans with AI is an efficiency guru and promising future leader (even if the AI sucks and winds up costing more than what it replaces).by beloch
- > I don't agree with the luddite-esque AI views like using ChatGPT makes you dumber and it thinks for you
I believe there are studies that indicate this to be true, however.
by King-Aaron - Honest question: what type of insurance covers a catastrophic failure when "AI" is implemented like this?
- I guess you have never sued the government, even a local one? Allow me to illustrate how this works:
Kids-for-Cash scandal: 2,500+ children were affected by thousands of juvenile cases in a corruption scandal centered involving two judges, private detention company (hired as part of youth services, and the juvenile justice system), thousands of police officers and at least half of school officials, and tons of social workers involved in the scandal imprisoned kids for a cumulative 417 years (that's the absolute minimum, they won't say how much it really is, probably a factor of that).
"resolution": ZERO restitution to the victims, the budget of the department responsible was raised by 10s of millions. Ciavarella was sentenced to 28 years (mostly for dodging the taxes on his bribes), of which he spend 17 in prison, while Conahan, who paid taxes on his bribes, spent less than 9 years in prison before home confinement, and had the remainder of his sentence commuted by Presidential clemency. He is living off his pension in Florida, and receives police protection (there's been several attempted murders of him by his victims. There is a slight issue with his police protection: apparently his police guard LET kids beat him up more than a few times too. He filed a case against the officers, which was dropped). To add insult to injury, extra budget was approved to you youth services "to help". Needless to say, they used part of the budget they got to imprison more kids for refusing to cooperate (and worse, they complain about the total unfairness that one of the employees was killed by one of those kids and the very light sentence that kid got for some reason). A LOT of people got attacked by kids in Pennsylvania for over a decade.
What they DID was imprison 2500 children for a minimum of 2 months each, bribed by money paid from youth services budget by a private detention facility. NONE of the other people involved faced any consequences. Not the private detention facility (children are locked up there at this very moment), not the youth services department. Not the people who organized the cases, not the mostly school officials doing the referrals, not the politicians approving the cases and budget that was used for bribing judges. In total about 1200 state officials were involved.
For comparison: Michael Webb kidnapping (Texas, 2019). A private individual confined an 8-year-old girl for about 8 hours before police rescued her. He was sentenced to life in federal prison.
In the kids for cash scandal 1200 state officials conspired to imprison kids for a cumulative total of 417 years, and a normal person gets life in prison for 8 hours of confinement. This would mean that a fair court system would have had enough cause to put EVERY state employee, and the governor who was responsible of course, in prison for at least 5 consecutive life sentences.
You see how holding the state responsible works? If it's bad enough they'll throw someone to the wolves, as well as everybody who refused to pay taxes. Totally ignoring that half the state's employees were involved, as a lot of politicians were pushing for this to happen (e.g. the governor was obviously strongly involved here). But if you compare the punishments for private citizens violating the law to what is given out if government is responsible for the exact same crime you get completely absurd comparisons. The punishments doled out to officials for the crime of kidnapping children were 46 THOUSAND TIMES less than to a criminal. That's 4,600,000% less.
Oh and to make the comparison even more ridiculous, the restitution the offenders owed to the victims, when they were state employees was MINUS 36 THOUSAND times what the court gave to a private individual's victims. And the result of this "justice" the state's budget was INCREASED due to the crime, with the net result of "justice" being more kids getting locked up. Again: government victims got ZERO (unless you count getting imprisoned again)
It's not just that courts are absurdly more lenient against their employer, they are also generous with other people's money and literally reward the offenders if they're the courts' employers.
(and let's not pretend Europe is any better. Every European government also managed to lead poison large amounts of kids, and failed to take responsibility)
So you see, there is no need to worry about the financial fallout when the AI fails catastrophically for insurance. No need at all.
by spwa4 - If you died, then complain.by GoblinSlayer
- AIUC (AI Underwriting Company) is the only thing I've heard of: https://aiuc.com/
They're new. We'll see if it works out. (Just did a quick search and there are some other new entrants too. Just found out about Testudo: https://www.testudo.co/)
At the same time traditional insurers are doing the opposite and limiting/removing liability from AI: https://www.ft.com/content/abfe9741-f438-4ed6-a673-075ec177d... (archive link: https://archive.is/ZfWyd)
by cootsnuck - You're seeing all these half baked AI adoptions in government specifically because they're the entity most likely to be able to dodge paying for it when they've done wrong.
- There are very strict rules for when the government allows you to sue it. If it's never been tried before it's usually because it's not allowed.
That said, wrongful death / negligence is typically suable and many policies already cover that.
by randyrand - I believe some places here in Washington are already using this type of solution. As someone with a relatively strong accent, to the point that it often confuses those cheap automated voice systems, I have mixed feelings about it.
On one hand, my accented English could be misunderstood. On the other hand, some of the more advanced systems might actually be able to recognize that I’m speaking Japanese right away, whereas a human operator may first have to figure out what language I’m speaking.
Considering that if I’m calling 911, I’m probably going to be panicked or at least dealing with something urgent, that could be one area where I see a real advantage.
A lot of LLM-based systems seem to be pretty good at detecting and handling many different languages, at least.
by unsignedint - I cannot read this article, but this may be partly confused what this function by Carbyne does. If there is a flood of critical systems, it just plays an auto message "If this is about X, thank you and we know about it" if you called from the same area.
I don't remember from prior articles if at that point you just stay on the line, or if you need to press a button. It is not going through an AI voice call tree.
by apwheele