Comments
Hacker News
another option for on-device routers is role-model that i'm building: https://github.com/try-works/role-model
by try-working
thank you! i find this very interesting, especially the Cache-aware concurrency
.
by eckelj
Personally I don't use LLMs, so I wonder what this kind of router is good for? Why not just a library?
by panzi
How do you decide when the cache-hit savings are worth riding out a degrading route versus eating the cold start on a healthy one?
by keynha
Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- another option for on-device routers is role-model that i'm building: https://github.com/try-works/role-modelby try-working
- thank you! i find this very interesting, especially the Cache-aware concurrency .by eckelj
- Personally I don't use LLMs, so I wonder what this kind of router is good for? Why not just a library?by panzi
- How do you decide when the cache-hit savings are worth riding out a degrading route versus eating the cold start on a healthy one?by keynha