Comments
Hacker News
Useful but how it compares with other providers model
Haha. somehow i just love the naming. it just makes sense :D
by XUEYANZ
this would be more interesting as a local LLM anlysis; throw out all the costs, and figure out primary-subagent model architecture, and maximize token generation and prefill.
I don't see how anyone can operationalize this information.
by cyanydeez
A cheaper model can look fine per request, but one weak answer may create another call or a human review step. That seems like the hardest cost to capture.
by mujia
I think this is great, and the next frontier is to analyze how well calls can be handled by a local model. Realistically, to do that usefully requires response time as a new dimension of judging: Can a local model provide an acceptably accurate response in an acceptable amount of time?
by pauldavis
This looks super clean. I'm curious about the --judge command. How does it evaluate if the cheaper model's response is a "tie" or acceptable? Is it using a specific LLM-as-a-judge prompt template?
by isadubois
Evals and OpenInference (OpenTelemetry) might be useful.
Costed opcodes (like the shelved eWASM opcodes cost chart) would be useful for this model routing problem as well.
Is this the cost to converge problem, the minimize cost to converge upon sufficiently low error problem, or the minimize cost and error problem?
EA methods: mutation, crossover, selection
Gradient descent as a mutation, crossover, and selection pattern; back up when the error/cost stops decreasing for too long and try a different branch.
A simple experiment: vary only a nonce in the prompt and compare output value. The nonce is a parameter. The model is a hyperparameter.
by westurner
What we need is an AI gateway/router that will actually first analyze the input tokens and then decide what model to use. If it's so simple that a dirt cheap qwen 3.5 flash or whatever will be fine, then it chooses that. If it deems we need GPT 5.6, then it uses that, etc. does anything like this already exist?
by indigodaddy
Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- Useful but how it compares with other providers model
- Haha. somehow i just love the naming. it just makes sense :Dby XUEYANZ
- this would be more interesting as a local LLM anlysis; throw out all the costs, and figure out primary-subagent model architecture, and maximize token generation and prefill.
I don't see how anyone can operationalize this information.
by cyanydeez - A cheaper model can look fine per request, but one weak answer may create another call or a human review step. That seems like the hardest cost to capture.by mujia
- I think this is great, and the next frontier is to analyze how well calls can be handled by a local model. Realistically, to do that usefully requires response time as a new dimension of judging: Can a local model provide an acceptably accurate response in an acceptable amount of time?by pauldavis
- This looks super clean. I'm curious about the --judge command. How does it evaluate if the cheaper model's response is a "tie" or acceptable? Is it using a specific LLM-as-a-judge prompt template?by isadubois
- Evals and OpenInference (OpenTelemetry) might be useful.
Costed opcodes (like the shelved eWASM opcodes cost chart) would be useful for this model routing problem as well.
Is this the cost to converge problem, the minimize cost to converge upon sufficiently low error problem, or the minimize cost and error problem?
EA methods: mutation, crossover, selection
Gradient descent as a mutation, crossover, and selection pattern; back up when the error/cost stops decreasing for too long and try a different branch.
A simple experiment: vary only a nonce in the prompt and compare output value. The nonce is a parameter. The model is a hyperparameter.
by westurner - What we need is an AI gateway/router that will actually first analyze the input tokens and then decide what model to use. If it's so simple that a dirt cheap qwen 3.5 flash or whatever will be fine, then it chooses that. If it deems we need GPT 5.6, then it uses that, etc. does anything like this already exist?by indigodaddy