Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- I'm sure many have made something like this, I've done a few. I've found simply submitting one's prompt to multiple models to be kind of pointless. You're just going to get statistical noise from the variances in their training methods, as they are all training on pretty much the same data.
I get significantly better results by pre-prompting each LLM (they can be the same LLM too, just another instance), I pre-prompt them to approach from a different perspective. Basically, I create expert personas that each believe they are someone of a different career, different intellectual perspectives, and then that generates a real debate between experts.
by bsenftner - Agree, and I see opus and Gemini pro as “quality” on openrouter fusion, this would be super pricy if the prompts are dynamic and not optimised for caching.
I would love to hear why they have created it, what was the business case, what this is going to serve? As you said, this is pretty easy to replicate
by Oras - I was reminded of "model alloys", where they randomly select a LLM for every agentic turn. This significantly boosted performance on security work.
(10 points on the benchmark, or a relative increase of over 20%)
https://news.ycombinator.com/item?id=44630724
TFA on the other hand tests two things at once: mixing models, and "fuse a model with itself",! the latter being just test time compute. e.g. Opus was able to match Fable on TFA, at the cost of costing twice as much money (and presumably time).
These two dimensions are orthogonal but can be combined for further gains.
It's not clear that every task benefits from it though. The only benched deep research, and their results are a bit weird. (e.g. they have DeepSeek outranking frontier models.)
More research needed!
by andai - I’ve been experimenting with two things on this:
- multi-model consensus, with multiple cross-review rounds. Obviously, the number of inference tasks explodes with the number of models. Led to some interesting results [^0].
- giving an agent "stray thoughts" produced by the same model, or another, giving the second model a selection of the agent’s context, with different triggers (random, loop detection,…)[^1]. So far has proven very helpful and much cheaper than the first.
[0]: https://github.com/lightless-labs/refinery
[1]: https://github.com/Lightless-Labs/skunkworks/tree/main/flux
by ElFitz - I have an old, slow GPU setup that has nearly 100gb of VRAM
I had been trying to fill this up with big models but it doesn’t seem like these give a good return per Gb
I’m looking at that and wondering would I be better off running multiple such models in parallel. It would probably be a better way to load balance across SLI.
My guess is the scaling will be more “mythical man month” than “no more free lunch” - the interaction of models resembling social dynamics moreso than multi-core setups.
Given that these actors are largely homogenous in culture and incentivising, and coordination overhead is drastically reduced.
Commonly we consider optimal team size to be between 3 and 7 and Brookes’ maximum team size is around 10 or so before the system fails. It should be possible to blow way past those numbers and still experience increased gains in productivity as long as you can keep all your instances stoked.
by rusk - I got significant improvement on code quality (so much that it has become a no brainer for important tasks such as planning) simply by adding the --self-review flag to swival: https://swival.dev/pages/reviews.html
Two instances of the same model, a producer and a reviewer, and the loops doesn't end until everybody's happy.
by jedisct1 - Similar feature launched open-source and end-to-end encrypted on my TrustedRouter https://trustedrouter.com/by ljlolel
- this is great
it's nice to see an actual privacy committment, i spend a lot of time reading through reams of evasive and nebulous provider terms
by lukewarm707 - Spent the weekend inspired by the new openrouter fusion model and wanted to see if it could run in Claude Code and if I could make it very easy for everyone else to try.
Built - claude-fusion-launcher — run Claude Code on a panel of models, not just one
Also shows cost
by SteveMorin - Doesn't it get expensive fast? I found the one-off prompts I did in their website to cost almost a $1/prompt.by admn2
- Some anecdata on Fusion: I run same query I used for Fable on OR Fusion and results were worse.
It felt, like Fable was able to kinda grasp very deep knowledge/intelligence layers and outline solution not only in agreeable way, but rather it proposed to prioritize solution items, with discarding some of the items, which made a lot of sense to me.
While Fusion felt more like a bit diversified answer of the same class of pre-Fable SOTA models, without touching the depth of knowledge/intelligence layers, which Fable was able to get, in my very limited tests I did, while Fable was accessible.
by arizen - I have been thinking a lot about this and my simplified understanding is that each model can be seen as a bell curve over human knowledge and each model has a different distribution. Using multiple models would allow us to change the distribution of other models with text that is out of their original curve. But then if you think about it does SFP and RL even alter the original distribution of text enough that models have enough variety so that their combined output is something better or just an echo chamber I believe not but I have no way to prove it yet.by alex7o
- I ran a quick eval to see what this looks like qualitatively vs just calling Opus 4.7 or GPT 5.5 directly.
As expected, Fusion was 7x slower and 4x the cost.
This isn't a knock against it, just that it I think this places Fusion into a "use it only when you need it" category.
- yeah its really counterintuitive i think; i.e, getting the right framework and structure for this to work probably isn't trivial, models really hate playing well together. i wonder how their version would fair in real world use.by galsapir
- Very cool, thanks for sharing. Speed was my biggest concern and they didn't mention anything about it.by awllau
- Which models were you using under this? If you used the quality default as exists in the interface, it makes sense that it was ~4x the cost as it'd be 3 frontier models judged by one of those.
The idea would be to use fusion with simpler, cheaper models.
by IanCal - Sounds like fusion would be a really good distillation target?by nielsole
- Context:
Surpassing Frontier Performance with Fusion
https://news.ycombinator.com/item?id=48525392
And a slightly better UI here: https://openrouter.ai/fusion
On OpenRouter's fusion API your request is routed to several models simultaneously and a judge model combines their answers into a final response. This significantly boosts performance, at the cost of time (at least on the one benchmark they tested, a deep research benchmark).
They have a Budget preset consisting of 3 cheaper models (which roughly matches Fable on that benchmark, costing half as much), and a Quality preset of 3 expensive ones (which beats Fable, but costs twice as much as Fable).
Pareto graph: https://openrouter.ai/blog/images/blog/fusion-benchmark-cost...
Curiously, fusing a model with itself also boosted performance (2xOpus4.8 roughly matching Fable on the benchmark, but costing twice as much as Fable). There's a further, smaller gain from mixing different models. The main gain seems to be from additional test time compute.
Would love to see more research on this, especially focusing on the cheap models that came out recently (e.g. Fusing DSV4 with itself, or with Mimo), and to see what the tradeoffs look like between running a fusion (parallel test time compute) vs increased reasoning or turns.
by andai - One of the things they neglected to discuss was how much longer it takes given the synthesis step. I guess for a deep research benchmark, it doesn't matter much, but will be interesting to see how it applies to coding tasks.by awllau
- I don't know if it is still the case with current models, but a few generations back Microsoft had some research results where asking a model to iterate N times would significantly improve performance, with the optimal point being 4 iterations.by jorvi
- Interesting how well a panel of Fable 5 + GPT 5.5 beats the frontier of either one, but if you add Gemini into the mix the panel of three performs worse, not better. To me that sounds like Gemini is worse at the given tasks but better at convincing judges of its solutions. Oh and a Panel of 2 Opus 4.8 models is almost exactly as good as one Fable 5. That smells suspicious. Do we know if that might simply be what Anthropic is doing behind the curtain?by sigmoid10
- > Curiously, fusing a model with itself also boosted performance
Back in the GPT2 to GPT3 era this was a pretty common thing to do. You are effectively taking more samples from the space of likely outputs. If your model can do the task 60% of the time just take 5-10 samples and implement some kind of majority voting
It became less common to use as models got high accuracy on problems where combining results is trivial. But with a more complex judge (a competent LLM) you can still get better results by just sampling more of the output space and picking out the best aspects
by wongarsu - I had a prompt I used for this just using Claude Code:
This works well with frontier models and even locally hosted models (last I used it was with Qwen 3.5).Let's review <filepath or specific file> for architectural issues. Spawn 10 agents, create personas for them, have them review the _api.go and write their review to reviews/<persona>-review.md, then have each agent do a round robin response to 3 of the reviews of their choosing (based on the abstract at the beginning of each review) and write the response to response/<original file name>-<agent persona name>-response.md. Then we do rebuttals to the responses in rebuttals/<response file name>-rebuttal.md. Finally, each agent should launch agents to review the reviews, responses, and rebuttals to their review, and compile findings to findings/<original file name>-findings.md. Finally, have another agent compile the findings and write that to review-findings.md. Present a concise version of the findings here.by all2 - Seems like a lot of machinery over-and-above running the same review n times + aggregating the result. What led you to the design?by kristjansson
- I'm new to using more than one agent in a flow so forgive my ignorance here but I have a few questions.
Do you review all the files that are generated to ensure there's no hallucinations? Do you just review the last file of concise findings instead?
Is the intent here that the hallucinations will be countered by running through multiple agents that you end up with only the truth? Have you seen anything in the last version that was egregiously wrong?
I was worried about the cost but if you are using local hosted models, then I suppose you don't need to deal with that as much. Locally hosted models still have issues running commands locally and reaching out to the internet right? So this is all just them running with the context of the file, without reference tot he rest of the project?
Thanks for any responses to this.
by daheza - Heh. I built "Fusion" a few months ago as an MCP using OpenRouter. The idea was to give Claude a "panel of experts" to go talk to when it got stuck.
After extensive testing and benchmarking I discovered that when you ask one model to judge another's response you don't actually get a better answer. You are just asking it "how closely does this resemble the answer you would have given me." Additional rounds and all the "obvious" solutions that pop into your mind reading the proceeding sentence are essentially just cranking up the temperature.
I did find a solution, but it is insanely expensive. Maybe if this gains traction I'll release mine.
by dsl - I think it depends.
I regularly ask both GPT and Gemini to give me options - programming libraries to do X, architecture suggestions, names for projects/services/classes
After they answer I ask each model what does it think of the other answer, and to give me a final suggestion considering both answers.
Both GPT and Gemini would frequently say "that other answer is much better than my one, it considered X factor that I missed".
by dist-epoch - My daily driver is Codex
I’ll have codex write a detailed prd.md file of how my project works, logic, goals, tech stack etc
Then I’ll use openrouter to have multiple models analyze that file, and write me a new improvements.md file
Then I’ll have codex check all those improvements md files and implement what it thinks are the best things that were missing
It’s quick, not expensive, and usually finds things I would have never thought of on my own
by r0fl - I've started to have different models review things like architectural planning docs- and I think for these more "fuzzy" outputs the differences between the outputs can be quite different and I can use my own "taste" to pick the best one.
I don't think it would work without a human in the loop but it is surprising to me how varied models' vibes are and how a system design varies by what it thinks is important to include and emphasize.
by awongh - Yeah, same experience. It turned out that objectively better answers were not that easy to find plus the expense plus it’s slow.by WhitneyLand