Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- There's an interesting bellwether in mobile games right now. There are two easy statistics to track when trying random free-to-start games: time to first ad (TTFA), time to paywall (TTP).
Most LLM-using mobile games are already at incredibly weak TTP scores and perhaps the only current stratum of mobile games where TTP is almost always ahead of TTFA. TTFA before TTP is almost nonexistent because they claim to need a monthly subscription as soon as immediately after install, despite being advertised as free to start. It's also one of the few types of games where the paywall explicitly does not include "no ads". Some of these games running on monthly subscriptions still need ads for unit costs.
(The bulk of mobile games try for a sweet spot of TTFA in the order of hours of gameplay and TTP in the order of days of gameplay. Easier to get people hooked on your game if you can give them a few hours of uninterrupted fun up front.)
It seems pretty condemning of software economics with LLMs involved.
by WorldMaker - This is one reason I've been trying to figure out tasks (and products based on those tasks) that can be pushed to the edge, either via small specialized models or small general purpose open models. I suspect the same desire to keep unit costs low is part of why Google is falling behind on the "frontier", but seemingly at the lead, or near it, on models that run on-device. I think they're just focused on making models for tasks that don't require boiling the ocean.
But, it's a hard problem. The models that run locally on normal computers/phones are pretty terrible compared to the frontier, without specialization and fine-tuning. And, even with specialization and fine-tuning, often a high-end general purpose model is going to do a better job and people don't need a bunch of local tools installed to do their various tasks.
by SwellJoe - "Users began expecting something fundamentally different from software: not just tools that store and retrieve, but products that reason, generate, and respond."
Absolutely not. Customers want systems for sales, reservations, accounting, and taking stock. That's where almost all the SaaS money is and none of it benefits from AI - and never will.
by carlosjobim - > Build the product once, distribute it to a million users for roughly the same cost as distributing it to one. Every incremental customer flowed largely to the bottom line.
common misconception about software unit economics. With enterprise software (one that costs real $$$) there always more costs attached post shipping. Before client/server it was support. Then it was security and the constant threat of cyber attacks.
Distributing software is nothing like distributing books.
by gofreddygo - >Users began expecting something fundamentally different from software: not just tools that store and retrieve, but products that reason, generate, and respond.
Not really.
>Every inference call costs money.
Not really, either. If you buy your own GPU, rack it, and run an open model, there is no unit cost. This is just expensive hosting infra. You also pay unit costs for SaaS that your software uses (things like SMS etc).
by roncesvalles - I think part of the new equation may also become; "Why even pay for the SaaS in the first place if you can just forge the service exactly how you want it?" The benefits of unlimited access to the tool you forge are still there, its just a lot easier to make whatever tool you want.
Are there any examples of products containing ai inference that are successful? Products that are beyond just direct access to frontier LLM's, I mean.
by smalltorch - I don't think it's at all certain this won't land back on the same unit economics as the old way. The cost of serving a user doesn't have to be free - it never has been - it just has to not be the dominating factor in your costs. I'm guessing there are still quite a lot of per-user costs that aren't easily visible. Like how many of your users are logging support requests, or suing you, or demanding bug fixes or custom integrations or a myriad of other things. And how much are you having to invest in security updates, regulatory compliance, marketing etc. Not to mention, users are getting well acclimatised to the idea of quotas and paying for increased limits.by zmmmmm
- "Inference" is just software running. It has always cost money to run software, it's just that it is generally too cheap to matter. If a client makes a regular API call to your server, you pay for that compute, probably in the form of a flat hosting fee. If too many calls come in and workload goes up, you pay for a more expensive hosting tier to handle it (or do dynamic scaling which is per-unit of compute).
Right now the "hosting" cost for inference is per-unit because it's new and expensive, but that won't last.
There is a lot of inefficiency right now keeping prices elevated. That will change very fast and soon paying for inference will likely resemble paying for hosting your app.
The bigger problem for SaaS is that the floor has risen - people can build their own solutions for things that they used to buy SaaS for. So the industry needs to level up and solve harder problems.
by chr15m