Birbla

Join the discussion

Write your take first — we'll ask for email only when you're ready to publish.

Sign in
  • Hacker News
  • I also tested it[0]: quite similar to GLM 5, a few percent better, 30% faster and 50% more expensive.

    [0]: https://aibenchy.com/?q=glm

    by XCSme
  • still 1/4 of the price of anthropic and openai models though
    by lousken
  • PS: Just added a cool feature, so you can filter the leaderboard for multiple models at once, by using a comma, like: https://aibenchy.com/?q=glm,claude
    by XCSme
  • benchmark where gemini flash is better than fable btw.
    by benxh
  • still quite verbose at 140m output tokens, but this is on max thinking. high should do better.
    by sourcecodeplz
  • Coding Plan is completely unavailable
    by linzhangrun
  • tangent question: Claude code seems to be very much loved and suggested by most major Chinese LLM using the env vars to change the server. that however means you lose a lot of anthropic tools like auto mode, running shells, monitors/crons. is there a way to get those with non anthropic plans?
    by fcpk
  • Would be interesting to see if we can make this model smaller with REAP + unsloth dynamic quant. It might become 4x cheaper to run for similar quality output
    by ttoinou
  • Local models are already useful today. The next milestone is getting this level of performance onto truly affordable hardware.
    by hemkeshr
  • NVidia has less than zero reason to ship cards ideal for this at low prices.

    AMD’s stock price reflects a hope they launch a CUDA alternative. But this is unlikely for the near future.

    There is a lot of interest in preventing China coming in with cheap AI hardware.

    So I expect the direction to be good local models that few can run effectively.

    by SV_BubbleTime
  • One or two more releases and they will reach Fable level.
    by DeathArrow
  • by then there will be Fable 5.21, again 5% ahead of every other SotA while still only 500% the size.
    by vitalyan123
  • Some more discussion: https://news.ycombinator.com/item?id=48567759
    by ChrisArchitect
  • I want to trust their benchmarks but when they have Muse Spark over GPT-5.5, it gives me pause.
    by theturtletalks
  • Where do you see that? I see they have GPT-5.5 (xhigh) at 55, GPT-5.5 (high) at 53, and Muse Spark at 43. Muse Spark does beat GPT-5.4 mini (xhigh) which scores 40, but the key there is "mini".

    In the coding index, GPT-5.5 gets 59.1, 58.5, 56.2, and 52.1 for xhigh, high, medium, and low while Muse Spark is behind at 47.5. For agentic, GPT-5.5 gets 74.1, 72.0, 69.4, and 59.7 (xhigh, high, medium, low) while Muse Spark gets 62.0 (beating only GPT-5.5 low).

    GPT-5.5 only gets beaten by Opus 4.8 in their general index, is the top spot for coding, and is #3 behind Opus 4.8 and GLM-5.2 for agentic (excluding Fable 5 which takes the top spot, but is unavailable).

    by mdasen
  • It's always nice to see how open source models growing, hope we will have good performance with lower tier hardware some day.
    by lanycrost
  • On our multi-agent coding and reasoning evaluations, GLM 5.2 is the first model we've tested that crossed the threshold of being on par with or better than Opus 4.6 (although as usual, we have GLM 5.2 and most other Chinese models a bit below most other benchmarks with test methodologies that are more vulnerable to benchmaxxing).

    Data at https://gertlabs.com/rankings

    by gertlabs
  • It does really well on "AA-Omniscience Non-Hallucination Rate", far higher than DeepSeek, GPT 5.5 or Fable. I really like that benchmark because it's one of the few benchmarks that allows LLMs to elect not to answer if they are unsure and punishes them for trying to bullshit their way through the benchmark
    by wongarsu
  • That one is a bit sus to me, because the models that do the worst on Omniscience Accuracy do the best on non-hallucination. The top model for this benchmark is "MiniCPM5-1B (Non-reasoning)" which gets a whopping 99% vs 45% for Fable 5.

    I'd love to see a good hallucination benchmark, but this isn't one. There's no possibility that a 1B model hallucinates less than Fable 5.

    by corlinp
  • This implies that other benchmarks (for which every AI provider is optimizing?) are actively encouraging bullshitting?
    by andai
  • It took me a while to figure out how to interpret the benchmark correctly, because on the overview page it says "AA-Omniscience Non-Hallucination Rate," but on the benchmark page https://artificialanalysis.ai/evaluations/omniscience#aa-omn...

    it said "the lower, the better." Eventually, I realized that the "non" reverses the scores. And indeed, the results are consistent.

    by SilverServer