Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- The problem with AI isn’t new, it’s the same old problem with technology: computers don’t do what you want, only what you tell them. A lot of PRs can be judged by how well they are described and justified, it’s because the code itself isn’t that important, it’s the problem that you are solving with it that is. People are often great at defining problems, AIs less so IMHO. Partially because they simply have no understanding, partially because they over explain everything to a point where you just stop reading, and so you never get to the core of the problem. And even if you do there’s a good chance AI misunderstood the problem and the solution is wrong in a some more or less subtle way. This is further made worse by the sheer overconfidence of AI output, which quickly erodes any trust that they did understand the problem.by prymitive
- Simple framework: You must not attempt to offload the actual thinking part to someone else. You should be able to answer any reasonable question about the code you submitted. If you can't you should be banned from any further contributions, permanently.by vrighter
- 2 months ago, after I started using Claude Code on my side project, within the space of days, I went from not allowing a single line of AI code into my codebase to almost 100% AI-written code. It basically codes in my exact style and I know ahead of time what code I expect to see so reviewing is really easy.
I cannot justify to myself writing code by hand when there is literally no difference in the output from how I would have done it myself. It might as well be reading my mind, that's what it feels like.
For me, vibe coding is essentially a 5x speed increase with no downside. I cannot believe how fast I can churn out features. All the stuff I used to type out by hand now seems impossibly boring. I just don't have the patience to hand-code anymore.
I've stuck to vanilla JavaScript because I don't have the patience to wait for the TypeScript transpiler. TS iteration speed is too slow. By the time it finishes transpiling, I can't even remember what I was trying to do. So you bet I don't have the patience to write by hand now. I really need momentum (fast iteration speed) when I code and LLMs provide that.
by jongjong - I dont mean to question you personally, after all this is the internet, but comments like yours do make the reader think, if he has 5x'ed his coding, was he any good to begin with? I guess what I'm saying is, without knowing your baseline skill level, I dont know whether to be impressed by your story. Have you become a super-programmer, or is it just cleaning up stupid stuff that you shouldn't have been doing in the first place? If someone is already a clear-headed, efficient, experienced programmer, would that person be seeing anywhere near the benefits you have? Again, this isn't a slight on you personally, it's just, a reader doesnt really know how to place your experience into context.
- > You can usually tell a prototype that is pretending to be a human PR, but a real PR a human makes with AI assistance can be indistinguishable.
A couple of weeks ago I needed to stuff some binary data into a string, in a way where it wouldn't be corrupted by whitespace changes.
I wrote some Rust code to generate the string. After I typed "}" to end the method: 1: Copilot suggested a 100% correct method to parse the string back to binary data, and then 2: Suggested a 100% correct unit test.
I read both methods, and they were identical to what I would write. It was as if Copilot could read my brain.
BUT: If I relied on Copilot to come up with the serialization form, or even know that it needed to pick something that wouldn't be corrupted by whitespace, it might have picked something completely wrong, that didn't meet what the project needed.
by gwbas1c - Shouldn't there be guidelines for open source projects where it is clearly stipulated that code submitted for review must follow the project's code format and conventions?by Bengalilol
- I’ve found LLM coding agents to be quite good at writing linters…by kasey_junk
- As if people read guidelines. Sure they're good to have so you can point to them when people violate them but people (in general) will not by default read them before contributing.by deadbunny
- > that code submitted for review must follow the project's code format and conventions
...that's just scratching the surface.
The problem is that LLMs make mistakes that no single human would make, and coding conventions should anyway never be the focus of a code review and should usually be enforced by tooling.
E.g. when reading/reviewing other people's code you tune into their brain and thought process - after reading a few lines of (non-trivial) code you know subconsciously what 'programming character' this person is and what type of problems to expect and look for.
With LLM generated code it's like trying to tune into a thousand brains at the same time, since the code is a mishmash of what a thousand people have written and published on the internet. Reading a person's thought process via reading their code doesn't work anymore, because there is no coherent thought process.
Personally I'm very hesitant to merge PRs into my open source projects that are more than small changes of a couple dozen lines at most, unless I know and trust the contributor to not fuck things up. E.g. for the PRs I'm accepting I don't really care if they are vibe-coded or not, because the complexity for accepted PRs is so low that the difference shouldn't matter much.
by flohofwoe - In a perfect world people would read and understand contribution guidelines before opening a PR or issue.
Alas…
by portaouflop - Code format and conventions are not the problem. It's the complexity of the change without testing, thinking, or otherwise having ownership of your PR.
Some people will absolutely just run something, let the AI work like a wizard and push it in hopes of getting an "open source contribution".
They need to understand due diligence and reduce the overhead of maintainers so that maintainers don't review things before it's really needed.
It's a hard balance to strike, because you do want to make it easy on new contributors, but this is a great conversation to have.
by isaacremuant - This is the thought that I always have whenever I see the mention of coding standards. Not only should there be standards they should be enforced by tooling.
Now that being said a person should feel free to do what they want with their code. It’s somewhat tough to justify the work of setting up infrastructure to do that on small projects, but AI PRs aren’t likely a big issue fit small projects.
by c0wb0yc0d3r - If one claims to be able to write good code with LLMs, it should just as easy to write comprehensive e2e tests. If you don't hold your code to a high testing standard than you were always going off 'vibes' whether they were from a silicon neural network or your human meatware biases.by quxbar
- Reviewing test code is arguably harder than reviewing implementation code because tests are enumerated success and failure scenarios. Some times the LOC of the tests is an order of magnitude larger than the implementation code.
The biggest place I've seen AI created code with tests produce a false positive is when a specific feature is being tested, but the test case overwrites a global data structure. Fixing the test reveals the implementation to be flawed.
Now imagine you get rewarded for shipping new features a test code, but are derided for refactoring old code. The person who goes to fix the AI slop is frowned upon while the AI slop driver gets recognition for being a great coder. This dynamic caused by AI coding tools is creating perverse workplace incentives.
by hoherd - I guess the main question I'm left with after reading this is "what good is a prototype, then?" In a few of the companies I've worked at there was a quarterly or biannual ritual called "hack week" or "innovation week" or "hackathon" where engineers form small teams and try to bang out a pet project super fast. Sometimes these projects get management's attention, and get "promoted" to a product or feature. Having worked on a few of these "promoted" projects, to the last they were unmitigated disasters. See, "innovation" doesn't come from a single junior engineer's 2AM beer and pizza fueled fever dream. And when you make the mistake of believing otherwise, what seemed like some bright spark's clever little dream turns into a nightmare right quick. The best thing you can do with a prototype is delete it.by jcgrillo
- Completely agree, I hate the “hackathon” for so many reasons, guess I’ll vent here too. All of this from the perspective of one frustrated software engineer in web tech.
First of all, if you want innovation, why are you forcing it into a single week? You very likely have smart people with very good ideas, but they’re held back by your number-driven bullshit. These orgs actively kill innovation by reducing talent to quantifiable rows of data.
A product hobbled together from shit prototype code very obviously stands out. It has various pages that don’t quite look/work the same, Cross-functional things that “work everywhere else” don’t in some parts.
It rewards only the people who make good presentations, or pick the “current hype thing” to work on. Occasionally something good that addresses real problems is at least mentioned but the hype thing will always win (if judged by your SLT)
Shame on you if the slop prototype is handed off to some other team than the hackathon presenters. Presenters take all the promotion points, then implementers have to sort out a bunch of bullshit code, very likely being told to just ship the prototype “it works you idiots, I saw it in the demo, just ship it.” Which is so incredibly short sighted.
I think the depressing truth is your executives know it’s all hobbled together bullshit, but that it will sell anyway, so why invest time making it actually good? They all have their golden parachutes, what do they care about the suckers stuck on-call for the house-of-cards they were forced to build, despite possessing the talent to make it stable? All this stupidity happens over and over again, not because it is wise, or even the best way to do this, the truth is just a flaccid “eh, it’ll work though, fuck it, let’s get paid.”
by corytheboyd - Essay is way more interesting than the title, which doesn't actually capture it.by jrochkind1
- Thanks for pointing this out—it made me take the time to find a sentence in the article body that could serve as a less baity title.
From https://news.ycombinator.com/newsguidelines.html: "Please use the original title, unless it is misleading or linkbait" (note that word unless)
by dang - The title seems perfectly engineered to get upvotes from people who don't read the article, which puts the article in front of more people who would actually read it (which is good because the article is, as you say, very interesting and worth sharing).
I don't like it but I can hardly blame them.
- So far I prefer the Hashimoto's solution to this that "AI tooling must be disclosed for contributions": https://news.ycombinator.com/item?id=44976568
I use it like this: If a PR is LLM-generated, you as a maintainer either merge it if it's good or close if it's not. If it's human-written, you may spend some time reviewing the code and iterating on the PR as you used to.
Saves your time without discarding LLM PRs completely.
by r4victor - But what does LLM-generated mean? What if I use CoPilot for completions? Is that considered "AI generated"? What if I grab the code from Claude, and update greater than 50%. Am I now taking ownership of it as my code?
It's like the ship of theseus
- > “I am closing this but this is interesting, head over to our forum/issues to discuss”
I really like the way Discourse uses "levels" to slowly open up features as new people interact with the community, and I wonder if GitHub could build in a way of allowing people to only be able to open PRs after a certain amount of interaction, too (for example, you can only raise a large PR if you have spent enough time raising small PRs).
This could of course be abused and/or lead to unintended restrictions (e.g. a small change in lots of places), but that's also true of Discourse and it seems to work pretty well regardless.
- Mailing lists are used as a filter to raise the barrier to entry to prevent people from contributing code that they have no intention of maintaining and leaving that to the project owners. Github for better or worse has made the barrier to entry much much lower and significantly easier for people to propose changes and then disappear.by warmwaffles
- Anyone else feel like we're cresting the LLM coding hype curve?
Like a recognition that there's value there, but we're passing the frothing-at-the-mouth stage of replacing all software engineers?
by softwaredoug - I was extremely skeptical at the beginning, and therefore critical of what was possible as my default stance. Despite all that, the latest iterations of cli agents which attach to LSPs and scan codebase context have been surprising me in a positive direction. I've given them tasks that require understanding the project structure and they've been able to do so. Therefore, for me my trajectory has been from skeptic to big proponent of the use, of course with all the caveats that at the end of the day, it is my code which will be pushed to prod. So I never went through the trough of disillusionment, but am arriving at productivity and find it great.
- It's been less than a year and agents have gone from patently useless to very useful if used well.by catigula
- Maybe, maybe not, it’s hard to tell from articles like this from OSS projects what is generally going on, especially with corporate work. There is no such rhetoric at $job, but also, the massive AI investment seemingly has yet to shift the needle. If it doesn’t they’ll likely fire a bunch of people again and continue.by corytheboyd
- When people talk about the “AI bubble popping” this is what they mean. It is clear that AI will remain useful, but the “singularity is nigh” hype is faltering and the company valuations based on perpetual exponential improvement are just not realistic. Worse, the marginal improvements are coming at ever higher resource requirements with each generation, which puts a soft cap on how good an AI can be and still be economical to run.by jandrese
- I feel like we need a different programming paradigm that's more suited to LLM's strengths; that enables a new kind of application. IE, think of an application that's more analog with higher tolerances of different kinds of user inputs.
A different way to say it. Imagine if programming a computer was more like training a child or a teenager to perform a task that requires a lot of human interaction; and that interaction requires presenting data / making drawings.
by gwbas1c - Well, when MS give OpenAI free use of their servers and OpenAI call it a $10 billion investment, then they use up their tokens and MS calls in $10 billion in revenue, I think so, yes.by dekoidal