Birbla

Join the discussion

Write your take first — we'll ask for email only when you're ready to publish.

Sign in
  • Hacker News
  • quantization level?
    by nubg
  • The article mentions Q4, Q5, Q8, and NVFP4. It's total AI slop though, tough read.

    In my testing I got 150 tokens/sec with a single 5090 RTX.

    by metadat
  • 5.01 BPW custom hybrid: bulk NVFP4, selected Q5_K/Q6_K tensors from an iMatrix, Q6_K embeddings and Q8_0 lm_head. The iMatrix was built from 5,472 messages across 296 real Hermes sessions
    by pich
  • they talk about the quants they tried in the article and settle on a Qwen3.8-27B-Hermes-iMatrix-NVFP4-Balanced.gguf which they calibrated on their own session traces and they pulled in 5 different llama.cpp pull requests to their local llama-server.
    by tingletech
  • Its egregious the quant level isnt disclosed along the "Qwen" string. Everyone knows theres huge difference in speed/quality along the quant axis, I now attribute the ommision of such to deliberate choice to not curb the hype of the tittle.
    by MaxikCZ
  • The whole site looks like and reads like AI slop. The outcomes also don't make any sense and don't feel rigorously tested (no, having claude test for you doesn't count as rigorous).
    by nodja
  • Please stop making this comment. The war is lost. Instead, you should be commenting that it looks like a human wrote this when you come across the rare brain-produced writing
    by IncreasePosts
  • The person is having a AI induced manic episode, we have all been there.
    by genxy
  • Why it's flagged?
    by gitowiec
  • Always put the quantisation in the title!
    by Tepix
  • Its not quite that simple here. The iMatrix-guided hybrid uses different quantization levels per tensor/layer, so there isnt one honest Q4/Q5/NVFP4 label I can put in the title
    by pich
  • Can you please try and see how many tokens you get with some form of concurrency. Pretty much ALL the benchmarks I've seen on the more accessible cards are just single request.
    by supermatt
  • Because concurrency with a single "accessible" card quickly diminishes. I have dual 3090s, and on Qwen 3.6 35B A3B at 80k a single card with max concurrent set to 4 will get 70-80tps single request, 50-60 TOTAL tps with 2, 45-50 with 3, and around 40tps with all 4 going.

    I know this doesn't exactly match your request, but I'm happy with my 3.6's performance. I've seen people claim they can get 120+tps on a 3090, but I'm not impatient when it comes to streaming text faster than I can read it.

    by jermaustin1
  • "Combining them into one heroic speedup would make a better headline and a worse benchmark."

    "The machine immediately taught me that capacity estimates are just admission tickets."

    "Useful in production, poison in a kernel comparison."

    Please don't publish writing like this, it's exhausting to read. You can edit that stuff out.

    The best part of this is the "Five xhigh artifacts from the finished model" section at the bottom, I suggest either moving that up or at least prominently promoting it at the top of the article.

    by simonw
  • lot of tech folks think their writing is boring and ask AI to make it more interesting or whatever

    llm gets them some of that sweet_llm_padding and readers smell the slop and attack the writing :)

    people, go write "boring stuff" - its more interesting!

    by mhaberl
  • Apparently we've blown way past the Turing test and approaching AGI and yet LLM-generated text still sticks out like a sore thumb. Maybe LLMs aren't that good at writing after all.
    by cedws
  • It has gotten so much worse over the last month. The default writing style of the Claude 5 model series in Claude Code is some sort of jiberish jargon.
    by PeterStuer