Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- Title is misleading (and is editorialized from the actual article title). GLM 5.2 did better than Claude in one specific cybersecurity-related benchmark (finding vulnerabilities of one certain type). I don't think you can draw any general conclusions about the relative utility of the two models.by kelnos
- 1000% this, this was us internally testing if our harness worked, the motivation was never to test them in-depth 1v1. We were just really shocked at the results, there’s a lot more work to do here.by insiderphd
- I've been using it for a week via opencode in a large, mature codebase for some moderately ambitious feature development, and a bit of debugging. Explicit purpose is evaluating if it may be a good substitute to save money for many tasks. For several tasks I've had both it and opus 4.8 attempt the same task and compared them.
In general, it's comparable across the board. Claude is less "verbose" -- GLM really likes to comment a ton. There were a few things where I think claude would have needed a little bit less back and forth. So opus still has an edge, but it's marginal, very much unlike previous open/competitor models where benchmarks looked good but actual day to day performance was pretty bad. I'm sure fable is "better" but it's so expensive + data retention policies are such that for the moment it was generally available I couldn't use it for work. This is still notably better performance than when claude code took the industry by storm.
I'm understanding why Dario is trying to regulate open weight models away.
by simplyluke - Are open labs just loss leaders backed by Chinese govt? Is this like electric cars where the goal is to flood the market with good enough quality for free so they end up dominating the market?
Or is there a business model I’m missing?
by softwaredoug - It's the same old "commoditize your complement" [0] playbook being run in the geopolitical arena.by gordonhart
- US EVs were also heavily subsidized, but they were all built using Chinese parts.by 34679
- > Are open labs just loss leaders backed by Chinese govt
There are many layers of Chinese govt. But GLM is backed by Beijing municipal govt and Tsinghua University.
by eunos - I use GLM 5.2 via Neuralwatt and it's gotten so cheap I wouldn't mind cancelling my personal Claude subscription if work gave me one. I've spent 374M tokens this month and it only cost me $18 on energy-based pricing.by jackdawed
- How's the reliability and speed?
- It reads like an ad.
Secondly these are "just" IDORs, arguably the easiest class of vulnerabilities.
Thirdly it compares to GPT 5.5 and Opus 4.8.
No, we don't have Mythos at home.
by danslo - Yeah they straight up say that their criteria is narrow and primarily important for their specific use case. Never let rationality cause your pitchfork to be cast away though!by jimbob45
- Technically we don't have Mythos at all? You guys have access. This tells me we have Opus at home (open weights).by sanid
- > Thirdly it compares to GPT 5.5 and Opus 4.8.
> No, we don't have Mythos at home.
That's still useful. To paraphrase the kids these days, GLM5.2 is in the room with us, today. Mythos is not. And for us in the EU, it's even more complicated, as Mythos might be with us in the room one day, and go poof the next day, on the whims of political entities that we have 0 control over.
Knowing where open, accessible, local models are is important. We know they're behind. But there comes a time when "good enough" is useful. Even if they're "just IDORs" today, and even if they're behind SotA today.
As someone else said above, GLM5.2 (and other models in the same tier like kimi, dsv4, etc) is / are slowly becoming "good enough" to assist in automated repo prepare work (download, install, test, edit, re-test, etc). And that translates in RL traces ready to be trained into the next generations. That might be more important than x% behind on benchmarks.
- In my experience, GLM 5.2 is extremely good at finding vulnerabilities, and more importantly, unlike Opus, I've never seen it refuse a command. It genuinely is a very strong model for finding and fixing vulnerabilities.
- >Thirdly it compares to GPT 5.5
mythos is <10% ahead of gpt 5.5 on all benchmarks, which it gains by being several times the size of opus. had it been economical to provide, it would've been released to the public on day one instead of the marketing circus those effective altruism clowns had exhibited. admitting that it costs >1000% to run inference on a <10% better model would've been very damning.
by vlian2088 - Has anyone compared the costs between maxing out a Claude Max x5 subscription (one for €120 euro a month) and same amount of work on GLM5.2 via API at a cost of $4 per mln token out?
I have a feeling Anthropic may still come out cheeper (mainly thanks to enterprises subsidising the Max subscriptions).
But I'm very excited with the possibility of using fully EU based inference rivalling Opus in quality.
by Roark66 - > [...] beating Claude Code (32%) at roughly $0.17 per vulnerability found
Claude Code is an agent harness, not an LLM.
Claude is a brand (or group of LLMs), not an LLM.
by WithinReason - It costs nothing to not be pedantic.by tills13
- The dollar amount is meaningless without comparison - and no other model has a price tag. Sloppy article.by croemer
- Claude code it's the only way to get access to the actual amortized cost of running a Claude-scale model. The consumer non-enterprise API is extremely expensive (with increasing marginal costs for the user and fat profit margins for Anthropic). If you want to approximate a State level attacker's cost where they can have the model on their own hardware, Claude Code is probably the best guess at the amortized cost.by Onavo
- Yes, and the article author is fully aware of that. Thank you for pointing out this small mistake though.by raincole
- GLM export controls incoming? I predict Commerce will force OpenRouter, HuggingFace to take some open models down within the next few months.
Not that it would make any sense.
by solenoid0937 - Obvious answer: build all your open source LLMs into firearms, get the SC to grant 2A protections.
- Cool then everyone will just change their config to route through a provider overseas for an added 50-100ms latency. Who cares.by dakolli
- I think state-of-the-art AI is going to be defense industry only from now on. We can have our toy drones but not the Predators and Reapers.by djeastm
- >GLM export controls incoming?
US imposing export restrictions on a model from China?
by gruez - > GLM export controls incoming? I predict Commerce will force OpenRouter, HuggingFace to take some open models down within the next few months.
I’m sceptical they could find the legal framework to do this even if they wanted to
They have legal authority to (a) prevent export of US goods/services; (b) ban imports of physical goods; (c) ban transactions (including purchasing services or license agreements) with foreign firms
But I’m not aware of any legal authority which lets them ban US firms from running a Chinese-developed open source AI model in the United States, if they are at arms length from the vendor, and aren’t using it for government contracts or regulated applications
Possibly they could order HuggingFace/etc to suspend Chinese accounts. But if someone in the US (or a third country) downloads the model from China then reuploads it to a US server, completely independently of the vendor - where is the legal hook to prohibit that?
by skissane - The Americans may ban the use of the Chinese models in America. But like the Chinese car ban, everyone else will use them.
- If that happens it'll be an absolute disaster. Imagine a scenario where Anthropic and OpenAI prohibit most US companies from using their latest models because of safety.. And meanwhile attackers use equivalent open source models to attack US companies.
Any prohibition on open source models will do nothing to fix the problem.. since attackers will never feel bound to the law. All advanced models must be available for defensive purposes.
by rgbrenner - These numbers are seem pretty low compared to what I was able to achieve specifically around windows kernel, win32k<->win32u to be exact. It honestly wouldn't surprise me anymore if china started surpassing models that US makes public, at least in specific categories such as cyber.
GLM 5.2 is already capable enough to assist in self-training which is similar to what we saw happen with frontier models and they appear to be getting there at a significantly lower cost than openai/anthropic.
by himata4113