Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- another option for on-device routers is role-model that i'm building: https://github.com/try-works/role-modelby try-working
- thank you! i find this very interesting, especially the Cache-aware concurrency .by eckelj
- No worries! Thank you for checking it out. The cache-aware concurrency was a ton of fun to implementby AndrewLiu96
- Personally I don't use LLMs, so I wonder what this kind of router is good for? Why not just a library?by panzi
- You don't use LLMs? Even my grandma use LLMs, what's your reasons?by addedGone
- How do you decide when the cache-hit savings are worth riding out a degrading route versus eating the cold start on a healthy one?by keynha