Birbla

Join the discussion

Write your take first — we'll ask for email only when you're ready to publish.

Sign in
  • Hacker News
  • > and use your traffic to (opt in) train you a model.

    Is there more info on this? I'm curious exactly what it is. Is it fine-tuning/LoRA on some base model? Don't cloud providers encrypt reasoning now - does that prevent this?

    by d2p
  • You started it a week ago? I look forward to checking back in 3 weeks when you've exited for $1B
    by 0xbadcafebee
  • See you soon
    by SilenN
  • Looks like first PR is June 24th: https://github.com/experientiallabs/experiential/pull/1

    So, two months. Still impressive!

    by tyre
  • Super interesting and congrats on the release. Curious if you initially had this in Python and then rewrote in Rust?
    by 23david
  • Yep! If you look at the commit history that's exactly what happened.
    by SilenN
  • We built our own model router at GPTree but this looks interesting. Caching is definitely one of the hardest parts to get right, especially in our scenario where users can branch from any part of the conversation and keep the dynamic context from the parent (unlike most other platforms that lock in the context once you branch).
    by foremerge
  • I have not tried it yet. Is it similar to LiteLLM? If so, what sets it apart?
    by cheema33
  • Router and model optimization from traffic is the main differentiator
    by kfallah15
  • >The gateway adds under 1 ms for BYOK requests

    Amazing! Really brilliant idea, thank you for sharing this project. There is so much ground to cover in the LLM gateway / routing / reporting world, and this is a great start. The Tinker implementation is my favorite part, fine tuning is much better than a sea of context files.

    by ceroxylon
  • Thanks! We are going to add continual RL via Tinker soon too
    by kfallah15
  • Open source and no markup is the right default for a gateway. The caching question above is the one I would want answered before swapping models though.
    by akshay_akula
  • Ans: we rarely switch, often times it's just a "switch to using this model for your agent"
    by SilenN
  • A part of me wishes the open source community would focus making research and industry-backed initiatives like the vLLM Semantic Router rock solid. Then I'd spend less time every month checking if this or that new model router has differentiating over vllm-sr :)

    At least for open source inference, it seems like there's healthy competition centered around vllm/sglang, but 2026 seems to be for model routers what 2025 was for agent harnesses.

    by nejch
  • Probably shouldn't call it "Open router" in the title as that's a specific brand. Maybe you meant "we built something like OpenRouter".

    Rereading I see you wrote "open OpenRouter", which looks a bit like a typo at first glance.

    by croemer
  • I think I have to look at setting up one of these AI gateways for work, since AWS bedrock is such a PITA to hook a harness up to over IAM roles. Also I still can't believe bedrock hasn't released any open models in months (so there's paranoia that I'll want to swap in another provider).

    Really though I'm hoping anthropic fixes the oppressive claude verbosity. I saw someone refer to being "clauderboarded" and my brain cannot let go of this as Claude's tokens bombard me.

    by jakswa
  • Oh. The UI screenshot on github is... not actually in the github repo? It's platform/hosted only? There's my first awkward discovery, but makes sense in retrospect.
    by jakswa
  • We had this exact problem so we solved it for ourselves. Happy to help if you run into any issues.

    I have a /hmmm command I use for the second part that works reasonably well: "Stop using jargon and speak coherently. State it more simply and concisely, like one human talking to another. Make it like google dev docs style. More dead prose. No aphorisms, no flourishes. Simple."

    by SilenN
  • Congrats on the launching of your product. I will be taking it for a spin to compare it with these other products that seem to be competing directly with what you have to offer:

    - https://github.com/ENTERPILOT/GoModel - https://github.com/maximhq/bifrost - https://github.com/BerriAI/litellm

    Would you care to share what makes experiential different?

    by mongrelion
  • GoModel author here. If you are looking for a self-reproducible benchmark, I run this one periodically. I keep it as honest as possible to know how GoModel stands in comparison to its competitors.

    Any feedback appreciated via GitHub issues: https://github.com/ENTERPILOT/ai-gateway-reproducible-benchm...

    by santiago-pl
  • Could you say more about how caching works? One major advantage of sticking with a single model is saving money on cached input tokens. I'd imagine if you swap between a bunch of models, you may improve performance but cost would would balloon out of control
    by Areibman
  • and caching is related to performance too ofc
    by purplecats
  • Generally you should only have two models in the pool per domain. I wrote some of my learnings building a router here: https://try.works/first-principles-of-model-routing
    by try-working
  • The trick is to rarely switch, or switch at task boundaries. Often the conclusion of routing is actually "this one model is actually at the pareto front for this task, just use it always".
    by SilenN