Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- That’s fun! I’ve been looking for something tiny I could embed in a webapp. Not a full genius model, just something light which could enhance the product without requiring ongoing cost.
Most organisations give their users terrible hardware, so anything which requires 32GB of RAM or a MacBook Pro won’t work if it’s a government or large organisation.
- I wish we could pop a tiny one into my phone so that when I type "Will see you" and swipe the word "later" it chooses that instead of "lasso"by viccis
- I like the concept and the development choices seem sensible even if they're not my favorites (although I think missing Python is a mistake). The text feels very LLM generated though, and I reflexively discount the value of anything presented with this writing style.by anigbrowl
- Shiny layer of marketing and proprietary code on top of open models?
Voz is Parakeet 0.6B v3
Clear is DeepFilterNet 3
Ear is the language predictor from whisper-tiny
...
by library8848 - seems to be…by sipjca
- but they're European!by joshuat
- AI beige theme and obnoxious AI writing signaled as much.by Muromec
- It is now also reasonably straightforward if you have access to frontier LLMs, a recent-ish mac and a recent-ish iPhone to point them at the job of porting a given model to run on the ANE - it's a reasonably easy task to hill-climb at this point!by sudb
- I recently had to use dictation for a few weeks and I was pleasantly surprised that many of the apps (in use/vogue) did support models on my 2021 16GB M1 Pro mac (many of those even supported connecting to a remote or local model endpoint) and at the same time for any worthwhile STT enhancement the model size was hitting higher I would have comfortably wanted. Even though I don't necessarily need dictation any more I intend to keep a custom fully offline setup and try these models (not sure they support live/streaming STT).
If any of you are interested there are apps like https://github.com/altic-dev/FluidVoice (this one's a great app) and this https://sam-pop.github.io/WhisperDictation. The latter, even though it has just 7 stars right now, seems to be more "intuitive". I just hope they expose a way to "connect" to available models on the machine or remotely)
by shelled - I love this idea and hope to see more on-device models. How do they make money, though?
I tried out their demo for Clear, the audio quality improvement model.[0] I'm not sure if it's just I don't have refined enough an ear or their demo is broken, but the "raw" and "enhanced" versions sounded exactly the same to me.
by mtlynch - The underlying model (DFNet3) is not particularly great but it is very small and fast - imo the best commercially usable denoising model is MossFormer2 (no affiliation - it's just excellent) with one drawback in that it can't remove reverb.
Nvidia's RE-USE model can do what MossFormer2 does _and_ can remove reverb, but it is non-commercial licensed.
by sudb - A lot of the models would be useful in a web context, to improve on the CMS we're making for clients. But they look like most of them are iOS only, few have a node package or something other, and all the benchmark are running it on modern iPhones so I doubt it would be that fast on a 20$ VPS.by ashenke
- Just a few are iOS first (pure practical timing/sequencing). We plan to make all models available cross-platform in the coming weeks.by pveugen
- at first i got very excited about a new fast transcription model (voz) but turns out its just parakeet v3 with some new inference code which is macOS/iOS specificby sipjca
- I was also thinking that this is almost too good to be trueby gorgmah
- Nice, now I know what I actually need.by Muromec
- Even worse, they made their additions proprietary:
> How much does Voz cost?
> Every model is free up to 100k monthly active devices per SDK. Unlimited inference per user. Contact us for custom licenses.
by yencabulator - This seems to be a common theme with these "European sovereign AI" companies. Mostly built on top of other open source work and slightly adjusted, often to collect grant money (although I'm not sure about that last part in this case).by splittydev
- It's an ANE optimized version of Parakeet, with our own inference, which enabled us to push performance to about 300x realtime speed on an iPhone 16/17. Our next gen Voz model is trained from scratch and will be at least twice as fast. Android and other platforms will land soon.by pveugen
- I definitely think there's a lot to be done with small models dedicated to specific tasks. I've always thought the REAL value is in having large models be able to easily build small models for custom tasks (which I know is kind of a thing), but perhaps just providing the small models directly is the more accessible approach.
>accessible via one SDK for Swift, Kotlin, and JavaScript
Lol well let me know when there's a Python SDK and I'll give it a try then. Obviously this isn't a deal breaker if you have a real case, but as someone who is willing to spin something up and try it out if there's a quick "pip install" command, this is getting put back on the shelf for now.
by nater5000 - Same, just wish I could afford the computer to train my own nglby ChickeNES
- The founders and website seems to talk a lot about mobile devices and how many are being sold/shipped, that's the "unused compute power" they're targeting. So obviously mobile-first platforms and SDKs seem to be the focus first, hence those languages. Understandable, given their target, but just like you I wish there was an easier way to give it a try on a desktop computer.
- > The world ships more than a billion capable phones, tablets, and laptops a year, most with a chip built for exactly this work, paid for and idle most of the day. Run the model there and the economics flip: no per-call cost, no round-trip, and nothing leaves the device.
This. I run small models (>50MB) for bio-imaging/biotech applications, it feels like every README implies that you need a discrete GPU to get started. While some do, many, especially the most useful ones, do not. Sure it matters if you're also going to do fine-tuning, but I believe your typical user just wants to detect some nuclei and get some cell-body ratios.
The laptop on your desk won't be running Meta's SAM, but it has more than enough compute to crunch 100's of your H&E slides overnight.
by momojo - I'm curious what models you're running and what hardware if you'd be open to sharing that info.by hypercube33
- This is a cool way of approaching AI models. I'm a big fan of local LLMs, local specific models like this should be even more powerful.
> Every model is free up to 100k monthly active devices. No tokens, no logins.
I dunno about the business model though. Cloud LLM billing makes sense: you're getting another computer to do work with each request, and using their compute via their gateway that they bill you.
These local models are like old school software. They're producing the weights, and then giving them to people. If I'm happy with the weights you've given me, and I'm not using your compute for inference and not wanting or needing any updates off you, why should you continue getting money off me and my customers?
The whole "but we need to keep it updated for your security" doesn't really work as well for software designed to run fully offline like these local models are.
I'm not saying they shouldn't get paid, but I guess I feel a personal sadness that it's less obvious how to successfully monetise such a sincerely useful and beneficial approach towards AI models.
by 1dom - > If I'm happy with the weights you've given me, and I'm not using your compute for inference and not wanting or needing any updates off you, why should you continue getting money off me and my customers?
What do you mean by "continue" if you haven't paid anything yet at that point?
by ricardobeat - > If I'm happy with the weights you've given me, and I'm not using your compute for inference and not wanting or needing any updates off you, why should you continue getting money off me and my customers?
Because that's the contract you agreed to?
If you don't like the contract they offer, don't take it. You can choose something else. It's really as simple as that.
- As far as I can see there is no clear pricing once you go beyond the free tier - it's contact sales.
This is actually a classic enterprise software approach - as there are no fixed per unit production costs to cover, the pricing is very elastic.
ie each license is probably a negotiation - if you see your app for a one off fee then they want a one-off cut - if you sell a subscription they may want a cut of that instead.
Obviously the risk is if you bake it in and your app becomes successful they may want a larger cut than you can afford, but if they don't negotiate in good faith they won't last very long.
by DrScientist - How would a model where you pay per each model/weight update feel like? More similar to traditional software, where you'd get the current version, maybe even free/rebated version upgrade to the next, but then have to do another purchase to get the latest version? Or something similar to that?
Feels like it'd be more aligned with how the business and users would produce and use it, it'd be something like: "We've done a bunch of research and output, here is the thing, you can buy it, and we'll work on the next, then you can pay again for the next if you'd like".
- > If I'm happy with the weights you've given me, and I'm not using your compute for inference and not wanting or needing any updates off you, why should you continue getting money off me and my customers?
Because they own the IP and they get to decide the terms of how it’s licensed.
This is like a company taking open source software, saying they like the code the community has given them, and asking why they should continue having to respect the terms of the license after downloading the code. The availability of the software (or models) does not equal a free license to use as you please.
A license allowing 100K devices for free is very generous. The businesses selling more than 100K units of anything will be significant operations. It’s fair that they’re asked to contribute financially.
by Aurornis