Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- > An engineer checked progress twice a day and reviewed EVERY proposed change.
press X to doubt [x]
by mparnisari - "Their old testing tool, Enzyme, had fallen out of active maintenance and was becoming a blocker to modernizing Asana’s frontend stack"
Non prod code, good use case.
- Also a very trivial piece of work for an LLM to do - with a measurable end goal.
Hopefully a human did give a quick look to make sure it didn’t just delete the tests.
by iamflimflam1 - They probably meant 1 employee part time for 5 years on a project that was never a priority.
- I can easily clear lots of low-priority engineering work quickly using AI. It takes a bit longer if I want to do a good job.
But here’s the rub: this is, basically by definition, low-priority engineering work. Those fixed papercuts are nice to have fixed but they do not necessarily add a lot of value.
And it’s all too easy to lose value by doing this. For example, current LLMs really really like adding test cases, and a lot of those test cases have basically no value, and carrying them around is not free.
by amluto - It has it risks too. I've used chatGPT for a feature I am working on. It works, I have already resolved some bugs the it didn't pick up. However, the code quality is odd, I've had some hardcoded types as it is unlikely they will ever change. For some of them chatGPT created enums but only for some.
I also realized that review generated code sucks, I can generate a whole app quickly, but I have no understanding of small decisions. It is extremely difficult to wrap your hand around hundreds of lines of code written by someone else.
by avgDev - Is this the react test framework enzyme? To me, $6 million and 5 years seems like a very high estimate for this even without any automation (AI or otherwise) for even relatively large applications. How could they possibly have a test suite this large?by lancebeet
- The more tangible example for me around LLM enabling what's previously long if not impossible tasks: pgrust. https://github.com/malisper/pgrustby tehlike
- From the article:
> For comparison: the previous plan was expected to take at least five years and estimated to cost roughly $6M.
If that's an estimate from an LLM, those have always been way off for me. I'm constantly amazed whenever something that an agent estimates would take weeks ends up being completed in an hour.
by ValentineC - The LLM is reasoning about estimates from its training data... which is to say, from human engineering timescales.
I suspect the labs could improve the models such that they are estimating these sorts of things but they don't prioritize doing so (or perhaps RLHF selects it away) because, as you say, it feels amazing to do a week's worth of work in an hour.
by aroman - I would love to know the responsible people at Asana who approved this joint PR. Like, do you not get the blatant second degree negative implications of what’s stated here, albeit manipulated and hyperbolic, for a project management software?
- If you ask Sol or Claude how much time it will take to implement a plan they just came up with, they usually advise a timeframe in the weeks or months - assuming, I suppose, that human programmers will be building it. And then you ask the model to just "do it" and it takes an hour or two. I always find this entertaining.by ttul
- I find that the models have increasingly started throwing out made up amounts of time around, as if they wouldn't be aware that the user is already using Claude Code or Codex.
It's like every time they make a plan, there's something about things taking "a week or two", "month of focused work", or whatever.
This is something that would've been RL'd out a long time ago if it wasn't great for business.
by helloplanets - Funny but shows how it doesn't have a useful world modelby kzrdude
- People may think it’s an overstatement. But I think it’s not.
This kind of projects are where AI is most helpful. Long tedious and highly testable projects like ports or legacy system replacements where humans have to grind through millions of lines of code without really thinking are the perfect target for AI.
I once did a C to C# port of several math libraries, and while I was able to automate most of it, it still required a ton of manual work. I bet if I had Claude, I could do what I did in 3 months in a day or two.
by d--b - This is what I was thinking too. Ports are probably where AI has the one of the biggest time savings over humans.
Although also agree, that even if human's did it, they probably would try to automate as much as possible and speed up development, and maybe that original 5 year estimate was if it was done completely manually.
by computerdork - Phrased this way, my aspiring product leader hat activates: that 5 years of work cannot be of much value to a company worth billions because a 5 year estimate is the lowest possible qualification you can make. Its a “will never happen” shelving.
all work is stack ranked against other opportunities. no matter how many teams help to parallelize, leadership basically comes down to ranking the stack correctly. Valuable work will be at the top by any means necessary. “mvp”, probes, task forces, code yellows, and so on.
edit: it’s even a backhanded compliment to agentic coding. Asana a public company considers a $12k outlay for something they’d never spend real resources on worth a case study =|
by apsurd - yeah this was my read.
either it was a hail mary 5 year change-the-market offering, or it was something that staff can "get around to" whenever they have the cycles, and would, at current rates, take like 3-5 years to do.
was the tool worth 12k? if it's not a line in capex or opex budgets it's value is $0
- Apparently the task was to remove dead weight. Haven't read what they replaced "Enzyme" with.
I did subagent based removal tasks a few times. These were the ones that required the least amount of input or thinking from me, because the requirements are abundantly clear. "Remove this part of the code without breaking any other part or by porting the tests done with it onto another system."
In these situations the code acts as the bookkeeping ledger itself, and coordination complexity is almost a no-brainer.
by luckystarr - Unfortunately, this is the type of statements we can't verify. I'm not sure why these types of news are still coming out when we all have AI at work.
Whenever someone does such a huge drastic change like this, it's by ignoring a large chunk of code that most people were afraid to touch for good reasons. Now, that code is gone, AI is celebrated, things will break, people will work very hard in the background to fix it, with no fanfare.
by firefoxd - It’s just sales claims. Do you see that “contact sales” button? You’re not the target of that. Deranged AI-psychotic C-suites with a two-digit IQ and too much confidence are the target.
- > people will work very hard in the background to fix it, with no fanfare.
This. It looks like AI companies have managed to use this for their advantage. Can't really blame them.
by chanux - > I'm not sure why these types of news are still coming out when we all have AI at work.
Because "we all" is a bubble, and many people do not have AI at work, or at least not the level of usage that many people here have the budget for at their company.
by kelnos - Exactly. You can simply close all open tickets with <WONTFIX> and claim that you've cleared 5 years of engineering work in 5 minutes. It doesn't mean anything.by selcuka
- > where afraid to touch for good reasons
This appears entirely unreasonable.
Normally the reason is not good. The reason is that unit testing is missing or that downstream effects are not entirely mapped out.
Exactly activities that traditional software developers are loathing because they are boring and mentally straining.
by tossandthrow - But you _can_ verify if you had bothered: https://asana.com/inside-asana/migrating-off-enzyme-2-weeks
Key statement:
> But at the rate we were going, we were still roughly five years from finishing.
Everyone has seen how this sort of thing comes about: it's meaningful work for engineering but never business/product critical so it just drags along.
Seems like this time around someone just went "I wonder if we could do it this way" and it worked. Perfect example of ditching sunk-cost and starting from scratch. Great outcome for them.
by cpinto - > I'm not sure why these types of news are still coming out when we all have AI at work.
Because OpenAI is burning $15 billion/year, and outrageous stories like those get parroted in the media. It's free marketing for a company desperate to get middle management to believe that a $500/mo subscription is absolutely crucial for every single employee.
by crote