Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- any plans to support https://openrouter.ai/ ?
- I like the pricing model but I'm skeptical it will last.by puppycodes
- What local models will it work with ? Also what will be the pricing for local llms?by Jayakumark
- interesting. is the answer not context specific most of the time? even if I ask LLM the same question again and again the answer depends on the context.
what are some use cases where you need deterministic caching?
by invisibleink - We often will repeat calls to try again. Or sometimes we make the same call multiple times to get multiple answers and then score or merge them.
Is this used only in cases where you assume the answer from your first call is correct?
by barapa - We spoke to a number of browser agent companies who said deterministic RPA with an AI fallback was their "secret" :)by rajit
- Funny, we are working to implement this same logic in our in-house financial categorization agent. When we have a repeat prompt it goes to a json that stores answers and only goes to AI for edge cases.
It’s a good idea
- I also did computer agents with a vc backed startup, ran into the same issues, and we built a fairly similar thing at one point.
It’s useful but it has limitations, it seems to only work well in environments that are perfectly predictable otherwise it gets in the way of the agent.
I think I prefer RL over these approaches but it requires a bit more data.