Birbla

Join the discussion

Write your take first — we'll ask for email only when you're ready to publish.

Sign in
  • Hacker News
  • The absolute best way to prove this works is by releasing a model that was fine-tuned with this method and then showing benchmarks depicting the improvement delta between the base model and the fine tuned one.

    The work is not done. Then release it to the masses and wait a few days for the actual real world anecdotes.

    Until then, this is noise.

    by Art9681
  • Yeah, this is just slop. No benchmarks, no concrete case studies, just some vibecoded "platform" to finetune models on your own traces.

    Which is an idea that has some value, but also some weaknesses. And this implementation of it isn't forthcoming with that concept. You have to really dig in to understand what they're even talking about.

    by Reubend
  • Benchmarks are the ultimate consolidation of halnons razor.
    by irishcoffee
  • Valid criticism. Happy to answer any qs. We're still working on solidfying results.
    by SilenN
  • Excited to play with this.
    by rglover
  • Let me know if you have any questions!
    by SilenN
  • Cool idea! How do you guarantee privacy?
    by yiyingzhang
  • It's open source!

    We do have a platform we'll be launching as well to manage training + serving for you which will require more diligent privacy guarantees.

    by SilenN
  • Cool project
    by digitaltrees
  • Thanks :)
    by SilenN
  • The title is misleading. This is model routing, not distillation.
    by surround
  • Fixed formatting which will help with readability. We do routing, distillation, and token compaction.
    by SilenN
  • Interesting approach. What does the cold-start phase look like for a new agent? How many traces or runs do you typically need before the router has enough signal to safely offload tasks from the frontier model??
    by anshad2u
  • Technically 0 because a) it ingests your already existing traces and does an initial training run b) in the app we'll have pre-trained routers you can start with that will then learn over time
    by SilenN
  • Probably a silly question, but ...: if it's optimizing itself purely inside a simulator, what keeps it from just exploiting the simulator's glitches to look good on paper, while not being so good when it hits the real world?
    by h2aichat
  • Great question. You need the simulator to be realistic enough that the optimizer cannot reward hack, but it does not need to be perfectly realistic.
    by kfallah15
  • Not sure I get it. The model you're improving is local? If so how do you even calculate cost compared to an API
    by jack_pp
  • Open source models.

    wmo routes requests between frontier models and open source models that continuously train using Tinker. As the smaller models improve, more traffic gets routed to them.

    Calculating cost is just tokens in/out.

    by SilenN
  • Local models need to be tuned to work well so this looks useful. Seems to be for general purpose model serving. I’ve been using https://github.com/adrianco/retort to run experiments for coding models across 13 different programming languages to see which frontier and local models work.
    by adrianco
  • That's cool, thanks for sharing!
    by SilenN