Birbla

Join the discussion

Write your take first — we'll ask for email only when you're ready to publish.

Sign in
  • Hacker News
  • looks like it's just qwen 3.6 coder.
    by cyanydeez
  • Do you mean it's based on qwen 3.6 coder?
    by SubiculumCode
  • its worse at code compared to qwen 3.6 coder.
    by lumost
  • strange, I already submitted the same url 6 days ago:

    https://news.ycombinator.com/item?id=48475095

    by AbuAssar
  • What's strange? Yours got no comments, so another attempt seems okay. It's pretty random what gets to the front page when.
    by mkl
  • Wasn't aware that Cohere was still around but this release doesn't exactly instill confidence.
    by zuzululu
  • Really? Why not. From the benchmarks at least it's a pretty decent small model.
    by kadoban
  • Aren't they focused on embeddings and strong there?
    by redwood
  • >Wasn't aware that Cohere was still around but this release doesn't exactly instill confidence.

    It's being kept alive because the Canadian government is desperate to have a local frontier lab and is willing to inject funding and force its adoption in government services, but leadership at Cohere is known to be weak in Canadian tech circles, and they pivoting to an enterprise-first market around production RAG rather than anything close to frontier work.

    I'm glad they're doing open weight releases but they're not viable in the long-run. It is embarrassing sharing similar spaces with them, but I'll try this release out in OpenCode and re-think afterwards.

    by greyb
  • I'm excited to see more OSS models
    by tonyrice
  • I was a fan of coheres general purpose LLM. Command A I think? Before they came out with their reasoning model.

    More competition is better.

    by moojacob
  • I always forget the VRAM requirements on these MOE things
    by SubiculumCode
  • Well, this is certainly not benchmaxxed, I'll give it that. And props for being honest about how far behind Qwen 3.6 MoE is this model.

    But yeah, it's not the best look to have to stretch and say it's "competitive" with other models in it's weight class, when it offers not much else that's useful or novel.

    by montroser
  • > Hardware (minimum): 1× H100 @ FP8

    Cool to see this but seems like it would be pretty expensive to run

    by matt_daemon
  • 4-bit quantized 30B-A3B MoE models can run at something like 21 tokens/sec on a several year old AMD CPU.
    by yencabulator
  • This is a 30B parameter model with 3B active. It should run performantly on a Mac with > 48GB RAM at 8bit precision.
    by anon373839
  • Are these models trained from scratch or do they necessarily need distillation from bigger models to be competitive? It's usually the case that they're a small model for a family with a bigger model. In the first case, does anybody know what's the economy of training this 30B-A3B model vs. training a DeepSeek V4 Pro or Flash size of models (1.6T, 200 something B, less activated)?
    by amunozo
  • You don't have to train from scratch but you can. Distillation ends up being somewhere in the ballpark of 1000x faster to train [1]. It also comes with the huge advantage of not needing to create RLHF datasets, since you can just copy the behavior of the teacher model. This saves an enormous amount of labeling money at the cost of making the model behave similarly to the teacher. If you are training from scratch, you can look at LLM scaling laws to figure out roughly the compute budget you need to optimally train a model [2].

    Based on [2] a 30B model needs something like 2e+23 FLOPS to train from scratch whereas a 1.6T model needs something like 1e+27 FLOPs to train. So DeepSeek v4 Pro was roughly 5000x more expensive to train than this model. I'm not totally sure how MOE affects scaling laws, so these numbers might be different in reality, but it gives you a good ballpark estimate of the difference in training scale.

    [1] https://arxiv.org/abs/2505.12781 [2] https://arxiv.org/abs/2203.15556

    by namr2000