Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- Pretty incredible performance for the footprint - really interested to see what could be done on slightly more powerful SBCs like some that have been mentioned in this thread.by kamranjon
- i don't care about microcontroller, what is the decent option to have local llm in my raspi4 that does not take 30 seconds to answer?by anonymous344
- While running LLM on tiny device is awesome, I'm more impressed by whatever training has produced the weightsby NooneAtAll3
- >esp32-s3
This microcontroller is a beast, currently using it to do dev work on a pi4.
Having two usb ports with one otg lets you do some neat things that would cost $100+ otherwise
by spacedoutman - Why can't this scale to run much larger models on CPU backed by flash with good access patterns?by chrishynes
- Voice-to-Text and Text-to-Voice models are approaching that size. I wonder how close we are to getting small devices that can chat with us.
Imagine a world where your toothbrush could give you tips about dental hygiene - or advertise toothpaste. What a time to be alive!
- This is a really neat use of the per-layer embedding trick. It's also worth noting that there viable TTS models that are ~20-30M param, so it might mean you can have a ESP32 with no network access read stuff out to you in near real time!by rao-v
- It's crazy what $5 can buy you in a microcontroller these days. Have a look at these Milk-V boards:
The duo has up to 256MB of memory, and a 1TOPS@INT8 TPU. They run Linux and are $5. I bought 5!
by titzer