Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- Pricing based on tokens always seemed a little weird to me.“Tokens” was and still is an engineering concept. The fundamental unit of transformer encoding and decoding.
But I have a sinking feeling that many AI developers think “tokens” got their name from the same idea as “virtual tokens in a casino” which is more related to product pricing and business.
by janalsncm - The Sonnet 5 comment is spot on. Even Anthropic's own graph initially showed lower performance at higher costs. Only thing I notice about Sonnet 5 is that it does appear to hand off tasks to agents more frequently similar to Fable, but of course nowhere near the quality of Fable. My guess is that Opus 5 will do similar but just isn't ready yet.by nathanyz
- Efficiency is the next frontier in LLMs, and I'm not confident the American companies are taking it seriously enough. DeepSeek, even in a naive API-calling loop, serves something like 80-90% cached tokens at an absurdly low price per token. Using an agent harness tuned specifically for their caching (Reasonix) pushes the cached tokens to 97-99%. DeepSeek is consistently among the cheapest models per task in my benchmarks, while also performing quite well. I'm still almost always using Claude for work, but for side projects, small stuff, etc. and anything better served by an API rather than starting up Claude Code (or `claude -p`) I'm using DeepSeek pretty often.
Anthropic models also shut down on a lot of security-related work, which is what I've been spending a lot of time on lately. I expected Fable to refuse this kind of task, but even Opus 4.8 refuses to build a verification harness for security bugs, as that involves exercising a discovered bug to prove it's been fixed in an automated red/green way, which looks like exploit creation to Opus' guardrails. So, I have to use other models for that work, now, though most of the original benchmarks I built were built with Claude.
by SwellJoe - In the context of local LLMs on limited hardware I've ran to the exact same conclusion: "tok/s" isn't the most useful metric when my personal North star metric, given my fixed hardware is: Model smart enough to execute my goals _in the minimum amount of time_.
Some models I tried (Mistral I think) had better tok/s, and roughly same billion parameters / scores on various benchmark... But they were _so_ verbose, that they generated many more tokens compared to a Qwen model of same caliber to answer the same thing.
So even though it had better generated tok/s, because so many more were generated, the clock time was longer.
And this compounds over mutli-turns: more generated token means more context used in the next turn (until some compaction or something runs)
by kpw94 - I feel we are caught in a "this is fine, pay more and we may turn down the fire" situation.
The LLM itself produces one token. Some tool adds that token to the input and runs it again, flogging the horse. Downstream another tool, some kind of harness, tries to control this stream by injecting tokens into the context and then sending it to the inference tool, and then trying to pattern-match the output.
Finally, there you are on CodePorn.yata paying for an agent to generate code, paying for an agent to tell you what's wrong with it, and paying for an agent to make it differently bad, and hopefully move on to the next task.
If it still hasn't dawned on you that this isn't just a bubble, but a snake-oil-bubble-bath, just try to imagine the paradigm shift whereby you go on github.com, assign an issue to an agent, the agent fixes it by rewriting the application in Pascal but a reviewing agent catches that you wanted it to print a measurement in Pascals (pa), and you don't pay for the work or the review, you only pay for work that one or two reviewing agents determine is up to par.
Nobody is going to do that because as soon as they test it they're going to have to do some math that won't make sense without admitting/realizing it's not some near-sentient, AGI rating 0.9 intelligence, it's just a text prediction algorithm that can pull out entire sentences when you use it to infer output on topics it trained on.
by kfsone - Cost per token doesn't say a lot, but "Cost per benchmark task" is also meaningless if your task is difficult enough that the cheaper model has no chance of cracking it.by yreg
- Cost per tokens is as valid as price per unit volume of fuel.
Changing the fuel type, efficiency of your vehicle, driving distance, or driving conditions will all change how much it will cost you.
Fuel cost per unit volume does not become meaningless just because you are neglecting all of the other factors involved. That would be throwing away the only data point you have been using.
This is just asking for someone to amalgamate all of the factors involved into one simple, easy to game, index.
by Lerc - I keep trying to convince directors and executives at my company to look past the cost per token amount but they refuse to do so. Those are the only things that actually give any sort of measurement of the monetary value of a token by these labs, and so its what many go by.
For example there's some benchmarks that show that Opus for any task that requires a higher than `high` level of effort, may have actually been cheaper to use Fable on low even though the cost per token is drastically higher
Similarly with GPT 5.5 vs Opus. They simply look at the dollar amounts the labs assign to each model and run with it.
But part of the issue compounds on the fact that there are many people who simply default to the smartest model/effort and don't actually vary their model per task. So in some sense I don't actually blame them very much.
by Jcampuzano2