Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- Tangent, but I have a theory that refactoring is one of the best symptoms of a healthy dev team.
It's partly that refactors themselves have benefits, but I think more that the benefits to refactoring aren't visible to something like product-owners, feature tickets, etc.
If teams are refactoring to ensure the health of the overall software, it's a tell-tale sign that developers are happy making recommendations for good software, and that those recommendations are being taken seriously.
I think Martin Folwer might have actually coined the term "software rot" - either way, as an issue it happens most severely when a team either aren't motivated or empowered to build their vision of high quality software. When a team can follow their judgement of excellence, that's usually a great sign!
(and yes, obviously this can go to far, there are probably some teams who rewrote all their stuff in Ruby then Node then Rust and now something else to be "agent native", but in the coorporate world, I see a lot less of this than teams who just don't feel like they have permission to improve things)
by benrutter - It’s interesting that cyclomatic complexity or cognitive complexity scales with token usage. A codebase that enforces upper bounds on one of these two also (potentially) helps AI agents stay efficient.
- Interesting that the amount of code remained essentially unchanged. In my experience it is not unusual that refactoring messy code cuts the number of lines in half.by danbruc
- Quality of lines > Quantity of lines.
In some projects, the code might not have much, or enough testing, or documentation/commenting.
Code is largely for others and the future if the creators of it want to move onto other projects.
by j45 - Yep, this was a real surprise to me, too. That’s why I included that top line in the figure. It also doesn’t line up with the agent’s predictions when it laid out the refactoring before proceeding.
There’s further work to do to understand exactly what’s going on here.
by gga - Yeah, I sorta expected the punchline to be "this 17,000 lines of code became 2,000" but it is still about the same quantity. Kinda interesting. Also interesting that the last change has by far the largest effect (4x reduction in tokens) and that also corresponded to the biggest reduction in single file LOC
- It can go both ways. Lines of code is a terrible metric for pretty much any goal that refactorings are meant to tackle.
- Agent code can only be read and understood by agents. We’ve reached the point where people just can’t comprehend these giant code blobs.
Feature, bug or emergent property?
I don’t think the distinction matters as much as the reality.
We’re being locked into using the AI tooling bc the code was generated with AI tooling.
These giant files of doom were being generated by humans anyway and were very hard to work with. With LLMs it’s at last manageable or feasible to edit, refactor etc.
I honestly think LLMs are going to save us from ourselves as the codebases became too large and “messy” for humans to comprehend. (Mono repos of doom)
On a personal level these giant files are abhorrent but that’s just personal taste and I don’t think any of the Martin Fowler refactor/cleanup stuff is going to matter at all anymore. Kinda sad on some level.
by pragmatic - Clearly not true. I got codebases that are 100% done by LLM. I have no problem reading the code base and find where things are. Not harder to find stuff than when I done it manually.
- > These giant files of doom were being generated by humans anyway
Right, I've never seen an agent produce code anywhere near as bad as some of the human-generated code that I've worked on.
And, if your agent is producing huge files or functions, you can just tell it not to and it'll comply.
by antonvs - There was always bad code. Extend that to the general challenge of a bad hire doing net-negative things for the company. AI is a new problem because it 1000x a bad hire's blast radius. Personally, I'd even state that AI tends to turn an average and sometimes-but-not-always-good hire into bad hires because finally they get to do the thing they've always wanted at lightning speed in a way that previously the company red-tape wouldn't allow.by apsurd
- > Agent code can only be read and understood by agents.
This just fundamentally isn't true and if this is your perspective then you're using LLMs wrong.
by Catloafdev - Great piece, but it misses the elephant in the room: the lion's share of economic benefit from refactoring will come from the fact that it makes it easier for humans to understand. That means 3am pages get resolved faster, fewer bugs will end up in prod, and your team can ship faster than your competition - gaining a leg up in the market. Most importantly, folks will be more comfortable accepting responsibility and ownership of a system when they understand it... this means that if/when something goes wrong, people will more quickly jump in and fix it, and when things could be better, folks will jump in and improve it.by holtkam2
- the punchline: "Refactoring reduces token consumption"
I appreciate the effort to quantify the benefit rather than pontificate. It's worth mentioning Martin Fowler wrote a whole book on refactoring [1], in which he states, "to refactor, the essential precondition is [...] solid tests", which I think is the real benefit here, AI or not. Good tests protect against regressions, whether human or robot. They also help encode the spec, which humans and robots can read.
[1] https://www.oreilly.com/library/view/refactoring-improving-t...
by sltr - I think the real punchline was that the value saved was on the order of cents!
> Every single change that touches the data access layer from this point forward now costs significantly less.
> How much of a saving? Assuming Sonnet 5 pricing at the time of writing of $3/MTok, 39.7 cents.
Now consider OpenAI's price drop, and open models, and consider that in the long run tokens will get cheaper. And think that the refactor needs to be human guided at a price of what for a senior developer - $100/hour?
by dash2 - (Just to note that although the article is on martinfowler.com Martin is not the author. It's attributed to Thoughtworks CTO Giles Edwards-Alexander.)by pmg101
- It's very interesting to have some data on this.
It matches my experience which is that LLMs greatly benefit from well factored code, but are not particularly adept at creating such code.
Much like most human developers I suppose!
by pmg101 - Out of the box, this is what I've found too.
I've had some luck while refactoring by helping it shape how to refactor, which can improve how well factored code should look like.
Providing examples of well factored code can go a long way, even if it's an open source repo of what to do / not to do.
by j45 - I've been budgeting myself explicit "slop removal" time. In fact I'm in it right now one window to my right here.
I still get a big win from AI on the net, but you do need to budget some time to clean up. I'm still on team "read every line".
In fact this is a case where I deliberately deferred some review because I was a blocker for another team. Now that I've got something to them I'm going back and I'm going to eat a bigger chunk of debt than I normally would, but it's worth it for unblocking the other team sooner. AI has made tech debt easier to take out, in all senses of that term.
It is also pretty decent, in my experience, at being guided into how to fix tech debt. Some other people's experience varies: https://news.ycombinator.com/item?id=49035455 YMMV.
by jerf - The benefits go beyond reducing token consumption. Compact contexts also foster better reasoning, enable intelligence across more layers if you can load them in a single context, and writing software to enable this, results in more correct software, software that _generalizes_, that has higher probability of being correct not just for tested cases, but for the interpolations and extrapolations of these cases.
Refactoring towards good abstractions is more powerful than people realize. There's information theoretic bayesian math to back this up.
It's a bit of a divine coincidence that software that is more economically and energy efficient to process and run tends to also be more correct.
It's all about reducing the entropy of your code. https://benoitessiambre.com/entropy.html
- Imagine what happens when human readability is discarded as a goal. LLMs are already very good at inferring meaning with little context. If the objective function is reduction of token consumption, it's hard to know where we will end up.by golemotron
- > It's all about reducing the entropy of your code
In all corners of our world and the universe at large, reducing entropy in anything can be thought of as “building.”
by hammock - I think this is one of those things where a human in the loop is indispensable
An agentic refactoring pass does make sense cause one LLM reviewing work can spot things the ‘generator’ LLM missed while focused on the initial task output
But can the reviewer agent ever actually have in mind what this project actually is? And how the code all comes together to do the work involved? In other words what parts of the code are redundant or can be made more elegant
Asking coding agents to refactor your codebase is maybe like asking trauma surgeons to increase your exercise capacity. The agents are gonna need a really holistic POV to do this properly
I guess part of my point is that just splitting big files into multiple files is only refactoring in a superficial sense without having a theory of what code belongs together and what can be extracted into utility functions etc. Is splitting files actually like decomposing factors or is it like splitting a larger number into smaller numbers that still eventually get added together
A good example of what I mean is that agents often don’t ~actually~ understand the whole system anyway. They might implement a system to store and calculate something that is already being fetched via API. Humans often have a dual perspective — a holistic sense of the project and (when applying our mind to a task) a precise scalpel: ‘oh if we just look at this this JSON it has a key with this data already’
by firasd - Your take is outdated. Agents are excellent at refactoring now. If you told me what you wrote during an interview, that would be grounds to not continue.
It's very important that you are using coding agents with the latest frontier models and know exactly what it can't and can't do if you want to be hired in this market.
by zuzululu - > Asking coding agents to refactor your codebase is maybe like asking trauma surgeons to increase your exercise capacity. The agents are gonna need a really holistic POV to do this properly
If you ask it to implement a specific kind of refactoring over a specific section of code it seems that current LLMs can do just fine. Even things as complex as "use `functools.partial` to implement the Command pattern here, rather than dataclasses".
> Is splitting files actually like decomposing factors or is it like splitting a larger number into smaller numbers that still eventually get added together
The file boundaries represent logical subsystem boundaries in the code, making it easier to reason about. The training data abundantly represents the idea of treating the contents of another file as opaque by default while other functionality in the same file can be used freely. I think it's reasonable to assume that there's something objective about the benefits that humans get from this, and it isn't just a consequence of how human cognition works.
by zahlman - I was able to largely refactor something that was going to take months in a week or so by being the narrating human in the loop.
It was helped by having refactoring experience and approaches to codebases by others, and in my case, being the original architect and being able to speak to the original and current intents, where needed.
This was using a less common, but capable and easy language for the LLM without a ton of dependancy brittleness to manage.. once the effort to remove javascript/python bias was in place, it became so powerful that once the lightbulb went on, it really got cruising.
The project was playing in the world of JSR-223 languages, where you could script in many popular languages, but it all got to run in the JVM, which was an environmental requirement.
https://en.wikipedia.org/wiki/Scripting_for_the_Java_Platfor...
by j45 - I really enjoy refactoring. Like, doing it by hand (5 miles uphill in the snow both ways) rather than AI. I don't even understand why I enjoy it, because if done right, there's no visible change. When people ask me why I'm so pumped up after having worked on the codebase all day long, I can't give them any sort of answer that makes sense (coming from a small business with a smaller dev team). It's just... "I'm future-proofing our website and we won't see any direct results today but things will be so much easier going forward..."
There's something about the puzzle. Looking at my old, deranged coding workarounds that tried to solve problems that have already been solved a thousand times before with established paradigms; and then moving them toward said best practices; and doing it in a way that no NEW technical debt is created. It's just satisfying.
I think one of the best learning experiences for me has been the fact that I created a bunch of sloppy shit by hand, auth and all, which forced me to learn things the hard way. All along the way, people were shouting from the rooftops: "use established libraries, dummy!", which is the same advice I would give to somebody today. But by doing things the hard way, I learned so much more about the inner workings. And, I've also given myself a decade's worth of refactoring work, which I really enjoy!
- I enjoy studying different refactoring patterns and real-world examples.by sqemo
- I'm curious how heavily you have test suites built up as guardrails while doing this, to prevent regressions and such?
- > I can't give them any sort of answer that makes sense ...
Pride of workmanship. For those who understand, no answer is needed. For those who don't, no answer will work.
by vogelke