GPT‑Live

Join the discussion

• Hacker News
• I'm crying, that guy who tries to get it to count to 100 may actually have a chance
  by rcarrol6
• I just told it to count from 1 to 100, and it actually correctly counted and didn't leave out any number. I'm impressed
  by nico1207
• Forget the fancy scores, the real test is whether it'll just let the guy count to 100.
  by nadzzz
• The ad with the grandmas is cute and funny, but from the first 20 seconds you can see that the voice annoyingly interrupts people while they are talking. It's almost as if it tries to reply too fast - faster than a real person would, and the results is that it replies while you're still talking.

  Oh and there was also a small fail in the live translate demo: the grandma says "tell him that..." which the bot translates verbatim, whereas a real translator would of course understand that this is an aside not to be translated.

  Well I guess at least I should be happy that they're transparent in their ads :)

  by OrangeMusic
• Maybe they are trying to outmatch the world's worst translator: https://www.youtube.com/watch?v=foT9rsHmS24
  by ndkap
• They don't act the way humans would.

  It's normal to start a sentence after the other person in the conversation pauses, especially if you're eager to say something (like the chat agent is), but a real human is socially aware and would in most cases immediately cease talking if the other person starts another sentence at the same time. Humans do this so much in their conversations that the other person doesn't really register the interruption at all.

  by simongray
• I'm so mad that this might make me re-subscribe to ChatGPT. I wouldn't have believed how much I use the voice feature before LLMs and ChatGPT currently has the best voice interface. I think Grok's interface is the next best, then Claude.
  by joshmarlow
• same, i use the voice feature every day at the gym, talk to it for 1 hour, and then make anki cards based on what it has taught me, total game changer.
  by misiti3780
• Just wait a few months and this will likely be available on a different LLM platform, especially if it actually works.
  by wraptile
• Same. I might switch back to ChatGPT from Gemini because I use the voice feature all the time.

  One of my favorite use cases is talking with it while driving on random topics and learning about them.

  by ksd482
• Are there any open source full duplex models that are out besides PersonaPlex? There was a chinese open one, maybe Fun Audio chat or something, that said it was going to release a full duplex version but I am not sure if it did.

  My dream would be open source full duplex with function calling or some kind of rudimentary text output. PersonaPlex is still interesting although it was looking like we would need to fine tune it to handle outgoing or avoid going off the rails easily.

  by ilaksh
• Working on this at https://duplexio.ai which will be open weights and free for non-commercial use. If you want to work on this send an email to anders@duplexio.ai
  by andersthuesen
• I want to know this too, as I’m hoping to fabricobble up a “smart speaker” that communicates with my local AI assistant. Right now we do everything via iMessage but it would be nice to be able to tell it to add things to my grocery list by voice while my hands are busy in the kitchen. Also would love if anyone has any advice on what microphone & speaker to pick out, was planning on just reusing a raspberry pi I’ve got around for the brain part.
  by VladVladikoff
• Standard Intelligence released one two years ago:

  https://si.inc/posts/hertz-dev/

  It's only 8.5B and doesn't sound like it's quite conversational.

  by programjames
• Gemini live has been able to do this for over a year now. I can just activate it on my phone and it really works surprisingly well, especially the interruption. I've tested it with my 95 year old Dutch grandmother and it switched seamlessly between English and Dutch with her and handled her poor hearing very well, including her asking for repetition.

  I'm a little surprised by how much OAI is playing catch up here.

  by ZeroCool2u
• What do you mean? OpenAI has had a real-time voice model since August of last year. This is a new model with better performance.
  by TaupeRanger
• Well Google has been working on voice assistants for years now on android. It makes sense that they had a head start
  by HDThoreaun
• Is gemini live the same app I have on my oneplus 15? I don't think that's full duplex?
  by davedx
• Exactly! I've been using Gemini Live for a while now and have been able to converse quite fluently. Maybe it's the long think functionality that's a differentiator here from OAI, which I'd be surprised if Gemini doesn't come out with of their own this year.
  by kkotak
• One thing Simon pointed out: Usually live voice models are not as capable as the frontier ones. What makes this one special is that it can delegate the task to a frontier model while talking to you.
  by BeetleB
• ChatGPT has had this for over a year too. This is a better model.
  by scosman
• Gemini live is also pretty decent at computer vision stuff too. I've used it while working on my bike & car a few times, with my phone camera and it'll circle/highlight specific screws and parts. You can set your phone up on a tripod and it'll walk you through complete repairs for things.
  by thewebguyd
• Much better than it was before but it’s still significantly weaker than a direct chat.

  For example I asked

  “Why should LLM attention use dot product instead of cosine similarity, being that we often hear vector magnitude does not encode most of the useful information needed”?

  The voice response was directionally right but lacked detail and was a little hand wavy.

  The answer to the same question in a text chat was much higher quality.

  The voice response replied “let me think about that…” so it appears to be invoking 5.5 as advertised, but it’s definitely weaker.

  I had reasoning set the same for both.

  by WhitneyLand
• I would naturally assume they cut the background thinking level to the minimum. It's a halting problem question and with in text 5.5 with max thinking will chew on a question for 5-10 minutes sometimes. That would make a pretty awkward conversation.
  by WarmWash
• You can read far more text than listening. If the voice response is too long, people lose the patience quickly. So it is better to show it on screen if we have too much text.
  by cnxhk
• A lot of this is steerable! Our current personality is optimized for brainstorming and conversations, but you can provide custom instructions to ask it to go deep and give you info-dense or more technical answers.
  by athyuttamre
• (Atty from OpenAI here)

  GPT-Live-1 is the first version of a new generation of models, and we believe the full-duplex architecture + delegation enables entirely new ways of human-AI interaction.

  Would love to hear your feedback!

  by athyuttamre
• Can it sing? Is it an end to end multimodal model?
  by vessenes
• I like it! I was watching a YouTube video of someone driving in a big city. I saw an interesting skyscraper and asked Chatgpt what it was. After Chatgpt answered, I asked for a photo of the building to confirm, and Chatgpt helpfully showed it in-chat. It was the correct building!

  And because the voice is so frictionless to talk to, I asked about what company owns the building, then that company's industry, then how that industry works in this particular country etc. I probably wouldn't have bothered going down a rabbit hole like this if I'd had to type. Voice is much easier than typing.

  Anyhow it's fun! Thanks for making it!

  by sandspar
• If it's not able to connect to my other apps like gcal, it should suggest that I use the standard gpt client, instead of telling me 'I don't have the capability to do that' which is misleading as some might presume the normal client doesn't have that capability either.
  by bariswheel