Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- While the tips are good to handle the volume, I still think this sets code owner on a dangerous path.
AI have limitation and hallucinate. Complex code will be explained in hallucinated way. At some point AI will be unable to write more because the arch has become too complex or the volume of code will be to high.
The article I would like to read would suggest how to force LLM to architect the code like a solid tower instead of a pile of unstable mud.
- I feel if you still architect the code and guide the LLM, it will do a pretty good job. Maybe one day the LLM will be able to do all the system architecture but that’s probably still quite some time out. I don’t even know if that’s possible considering different business needs and other factors that aren’t technical.by vjvjvjvjghv
- I love the idea that understanding is the new bottleneck. Because if we just ignore the potential horrors of cybernetic augments, it suggests the next challenge is how to teach things better. And that’s such a valuable thing to improve.
I have a soft spot for when I find a teacher or textbook or interactive website that makes something click. I live for that click. I crave it. I crave seeing it happen in others. How optimistic I could be if understanding becomes the primary target.
by Waterluvian - Would you have any good such resources for “clicking” or expanding one’s understanding to share?
From the top of my head and of my Goodreads I have enjoyed The art of Electronics, Understanding Earth, Material World, The world for sale, Beej’s Guide to C, The Five Dysfunctions of a team, Financial Intelligence for Entrepreneurs, The Lean Startup, Fouché by S. Zweig
- I found the linked article "books don't work" from the quiz part highly interesting
https://andymatuschak.org/books/
It explains a lot and works really well.
I tried out in ChatGPT with a simple prompt:
> ...paste link... Give me series of quiz see if I really understood the article well. Ask & answer one by one in turns.
Really fun experience.
by est - Thanks! Great readby augment_me
- It's been submitted many times before: https://hn.algolia.com/?q=books+don%27t+workby zahlman
- I am surprised by the title and the story. Understanding has always been the bottleneck; there is nothing new about it. The argument goes like we humans should understand so we can verify and participate. How bold! Maybe we should have been doing that all along...?
- A good chunk of the non-tech-savvy management equates coding with typing. They're rediscovering Amdahl's law in the agentic LLM era.
- Given it is from "AI Engineer conference" that is expected, it is like TED talks of LLMs.by maxnevermind
- For me the solution has been to throw away the code I don’t understand. I let the agent write the code, and if when I read it it seems unclear or needs a lot of explanation from the agent, I just throw it away and start over, or do it by myself.by hk__2
- This seems like one of the hardest parts for everyone to adjust to (including myself): code is now easy come, easy go. Or in other words, everyone seems to get trapped in the sunk cost fallacy, even if there's not even that much sunk cost any more!by xxpor
- Something funny happens when you reject code. The agent realises it is garbage.
I'm personally of the opinion that the tools for reviewing LLM generated code are awful. 99% of the time I want to do line by line comments and tell it everything it did wrong. Given that information, the next iteration would be much more up to my standards.
The same also applies to other people's LLM generated code. Yeah sure they can just pass on the comments to the LLM, but that will just mean more iterations and them losing their job.
by imtringued - This is where I’m at as well. If it needs to change too much to do what I want, I take that as a sign to either (1) ask for a change that is easier to review, or barring that (2) pivot to refactoring the codebase until it becomes a change that is easy to review.
Ironically, even in this era of cheap and instant code, what works best (for me) is still to write as little code as possible.
by evenhash - I am so dying to read more about the new/current/real bottleneck!
Where is the bottleneck? WHERE?? Tell me! No evidence needed, just lay it on, man to man, thought-leader to thought-leader!
by iainctduncan - Oh wait, there it is, sitting over top of the fat part and under the cork...by iainctduncan
- uncovering bottleneck is the new bottleneck!!
- It doesn’t matter, once you found the bottleneck there is a new one. Seems we changed the supposed bottleneck of writing code (as if it ever were, the world was producing far too much code before LLMs were even a thing) with about ten or so new ones, was it a good trade?by techpression
- > thought-leader to thought-leader!
This is my new chat-up line at networking events.
by euroderf - In the end, LLMs create garbage code that no one understands, they break things that should not have been broken, that would not have been broken if it was done slowly with understanding along the way. To reframe it as "understanding is the bottleneck" is just more LLM salesmanship. LLMs have their limits and when you hit them you're stuck. LLMs are the bottleneck. But the idea that "LLMs are the answer to the problem created by LLMs" is absurd.by nphardon
- I couldn't agree more. We really need more engineers to be vocal about how stupid these ideas are. "How about we throw out 30+ years of software engineering literature so we can 'move faster'?" What if the customers on the other end don't want new features, they just want software stability? If SQLite came out with a new LLM-written feature a week, would it be a better library? If you know what to build, writing software right the first time pays for itself over time. LLMs are still great, but more for rapid prototyping, researching, log diving, one-off scripts...
Sorry, </rant>.
by equinumerous - "I read the code." -Mitchell Hashimoto
Great code needs great understanding and agents need excellent guidance. Even in my current solo-dev work, I can't imagine making a production commit I haven't read until I understand it. I own the consequences of my code; that's a responsibility AI agents can't take.
by euthymiclabs - Conversely I'm a solo-dev and I ship a lot of slop I don't even look at. Granted my work is just basic CRUD apps, and I focus my efforts on validating important consequences (like does this break accounting invariants or something)by mindwok
- Yeah, but in the corporate world now, you're being judged by how fast you get shit checked in. So, if you need to understand every line these days... you're going to end up in the bottom 25% of the company pretty quickly. Not saying you're wrong, just saying how fucked things are becoming.by hackmack10
- Agreed with Reading! This alone isn't enough though. PreLLM too it wasn't just reading code to review. Someone did the hard work or crafting the code and each unit test would tell you the weird corner cases to deal with and factor that into changing your code. One person owned a part of the codebase and was an expert. Not to mention reading isnt easy when the velocity of code pumping in is 3-5x more. Its exhausting and reading becomes skimming.by conqrr
- It's more like this:
We've always lacked understanding. However, it didn't feel like a bottleneck; in spite of lacking understanding, we developed huge, complex systems that became hard to maintain and that nobody understood completely.
Now we want to scale that orders of magnitude, but when we do that, we feel the pesky lack of understanding.
We previously worked around the lack of understanding by making the system gradually incomprehensible in small increments, upon each of which we observed it still working, more or less.
If the whole thing materializes in one day, that doesn't work; the approach is gone.
You can now bring into being something which statistically resembles the old kind of system that was iteratively evolved. But the thing has no such history. You can't go back to play archaeologist. It looks like something that would have had users, but it never did. It was never in production anywhere. Nobody ever submitted feedback, or a bug report, such that it was fixed or improved. There never existed a simpler version of it that several ex-maintainers understood perfectly; there are no such ex-maintainers and no such understanding. There is no documentation trail, or other historic trail if surrounding activity like discussions and negotiations which led to things being the way they are.
by kazinator - I agree with the problem but not the solutions.
The problem pre-dates LLM's: writing code that "works" but breaks the underlying model. Because it works, it always sounds reasonable and doesn't raise any flags.
Only someone - human or LLM - who holds the model as the standard would see that this working solution breaks the model.
(In theory, the model is to preserve scaling, flexibility or some other systemic feature not immediately invalidated by this working code, but as always the model itself could be bad.)
LLM's are not bad at giving an account of the model; indeed, fighting with the LLM over what the model is can clarify things. But LLM's will happily hold on to a stream of inconsistent statements as their model, so they are not the authority.
by w10-1 - understanding a different modality of model interaction gave me proper insight into the specific problem. In visual models, even if the model understands the concept of face, or hand, or whatever, it doesn't know how to de-dupe a statement like "count the number of faces" until you give it a countable reference frame, so it can internally, place a box around a face and give that a coordinate, and then it can collect all the coordinates, and suddenly it's counting face in a picture.
The same thing happens in code. Things we're happily shifting from context to context, the model itself isn't doing. When it reads file1 for the main() clause, it will easily read file2's main() clause as the same. It'll internally merge these.
So if you do want to work with these models to achieve complex tasks, you basically do have to go reverse centaur and bend the code base to it's blindness. You can't use the same function names across the code base; each one needs to be dstinguishable; same thing with variables that represent seperate entity relationships.
You do that, and it suddenly because a whole lot smarter.
by cyanydeez - This is a very insightful comment.
I will add: The model is often completely implicit in the code. Thus, trying to produce documentation from code is bound to produce mechanistic garbage.
I see two ways out. Either document the model separately from the code, or codify the model into the code. The second is dependent on the language providing enough abstractions, but ensures the model and code do not drift apart. And I think in an LLM heavy setting, this will pay off.
by black_knight - We have LLMs try to generate descriptions of PRs for us and they're pretty universally disliked. They're always overly-complex descriptions of the mechanical changes and have no sense of motivation.
Also, a huge reason to understand the code yourself is to make sure the LLM isn't wrong, but this doesn't work if an LLM is itself generating the understanding.
by alecbz