Discussion summary
A 7MB in-browser embedding model called Ternlight was discussed, highlighting its potential uses and concerns about browser resource usage.
What the discussion says
- Some users expressed concerns about malware and memory hogging.
- Others emphasized that downloading resources is standard web behavior.
- Several users praised the technical achievement and potential applications.
“This could be used to distribute malware and also or hog excessive browser memory.”
“That's... how the web works? You download things on demand.”
Comments
Hacker News
Thank you for this! Local models will bring privacy at some point, and I already know an excellent use case for such a small embedding model (cheap and fast search in a product base). Relying on the CPU is also a plus in my case.
What we need is a W3C LLM API like the one Chrome already offers: https://developer.chrome.com/docs/ai/built-in
by esafak
I added an offline search engine to app.wazzup.im/search (no login or payment required).
First search downloads the model from the internet and subsequent runs are from the cache.
The model is very small so it's not the best for everything but it's good for basic math and coding.
Give it a try.
by wazzup_im
Can the 30 second embedding time be done beforehand and sent to the browser?
Inference is nice and quick after that.
by aetherspawn
This would be nice as an Astro (or generic meta-framework plugin) that automatically parses all generated html files and generates a small db of embeddings.
This way on the frontend you can lazily load this. Maybe you could even store the HNSW in chunks and just load the pieces you need for your specific search query.
i.e. like https://pagefind.app/ but to get fully static vector search.
by aziis98
This would be a pretty cool addition to the duckdb HNSW search project I found on here some time ago: https://github.com/jasonjmcghee/portable-hnsw
What I think is really cool is that the search happens using http range queries across statically hosted parquet files.
I think things like this could bloom into a relatively open and distributed search ecosystem that isn’t controlled by major corporations.
by kamranjon
This is cool!
but also maybe you could put a button on the landing page to trigger the demo because it's a bit startling to hear my fans go crazy when opening a webpage.
by dirteater_
Hobby project, I wanted to "ship a useful model in a web browser". so I distilled a small sentence encoder from MiniLM with ternary quantization-aware training. Also wrote the inference engine from scratch and shipped in Rust → WASM SIMD.
It's an embeddings model, not an LLM: text goes in, a 384-dim vector comes out, and cosine similarity between two vectors tells you how related the texts are — regardless of shared words ("reset my password" ↔ "I forgot my password" → 0.88). Used for semantic search, FAQ/intent matching, and clustering. Running it on-device means search-as-you-type semantic search is performant with no API dependencies.
Demo (2k React docs, fully on-device): https://ternlight-demo.vercel.app
Two tiers on npm: - @ternlight/base (7 MB, ~5 ms/embed, more capable embedings) - @ternlight/mini (5 MB wire, ~2.5 ms/embed).
Bundled for Node and browsers.
Repo - see technical details (MIT, training pipeline included): https://github.com/soycaporal/ternlight
Curious if this is something useful, what are the use cases for on-device embeddings.
by soycaporal
Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- Thank you for this! Local models will bring privacy at some point, and I already know an excellent use case for such a small embedding model (cheap and fast search in a product base). Relying on the CPU is also a plus in my case.
- What we need is a W3C LLM API like the one Chrome already offers: https://developer.chrome.com/docs/ai/built-inby esafak
- I added an offline search engine to app.wazzup.im/search (no login or payment required).
First search downloads the model from the internet and subsequent runs are from the cache.
The model is very small so it's not the best for everything but it's good for basic math and coding.
Give it a try.
by wazzup_im - Can the 30 second embedding time be done beforehand and sent to the browser?
Inference is nice and quick after that.
by aetherspawn - This would be nice as an Astro (or generic meta-framework plugin) that automatically parses all generated html files and generates a small db of embeddings.
This way on the frontend you can lazily load this. Maybe you could even store the HNSW in chunks and just load the pieces you need for your specific search query.
i.e. like https://pagefind.app/ but to get fully static vector search.
by aziis98 - This would be a pretty cool addition to the duckdb HNSW search project I found on here some time ago: https://github.com/jasonjmcghee/portable-hnsw
What I think is really cool is that the search happens using http range queries across statically hosted parquet files.
I think things like this could bloom into a relatively open and distributed search ecosystem that isn’t controlled by major corporations.
by kamranjon - This is cool!
but also maybe you could put a button on the landing page to trigger the demo because it's a bit startling to hear my fans go crazy when opening a webpage.
by dirteater_ - Hobby project, I wanted to "ship a useful model in a web browser". so I distilled a small sentence encoder from MiniLM with ternary quantization-aware training. Also wrote the inference engine from scratch and shipped in Rust → WASM SIMD.
It's an embeddings model, not an LLM: text goes in, a 384-dim vector comes out, and cosine similarity between two vectors tells you how related the texts are — regardless of shared words ("reset my password" ↔ "I forgot my password" → 0.88). Used for semantic search, FAQ/intent matching, and clustering. Running it on-device means search-as-you-type semantic search is performant with no API dependencies.
Demo (2k React docs, fully on-device): https://ternlight-demo.vercel.app
Two tiers on npm: - @ternlight/base (7 MB, ~5 ms/embed, more capable embedings) - @ternlight/mini (5 MB wire, ~2.5 ms/embed).
Bundled for Node and browsers.
Repo - see technical details (MIT, training pipeline included): https://github.com/soycaporal/ternlight
Curious if this is something useful, what are the use cases for on-device embeddings.
by soycaporal