My current strategy is to not read any of the code written by my agents

twitter.com44 pointsby SantiDev43 comments

Comments

Hacker News

Right now there is about a 95% chance that when I manually review an agent's code I find nothing that needs changing.

I find it plausible that an extra agentic review pass and more testing can bring this number up to the point that one never needs to review code again. AI writes pretty good code nowadays.

(You still need to be diligent and decide the architecture during the planning, and read the gotchas and "things to note" that the agent will spit out at the end of implementation if it had to diverge from the plan.)

by 2001zhaozhao

My current code base has more tests than runtime code. This sounds far more thorough than that. If you limit the possible productivity improvement to a small portion of total effort, it's inherently also going to be a small improvement.

Which, is fine, but not what most people are hoping for with these tools.

by nitwit005

>My current strategy is to not read any of the code written by my agents. That’s the only way I can take advantage of their productivity.

When you start getting good results from agents you soon realize you are the bottleneck.

Automating the verification of the code is the way to go, otherwise it's just not worth it. It takes longer to read and understand code than to write code, so why bother with agents if you are going to manually review it all anyway?

One thing I miss after ditching Copilot (it got too expensive) was how I could trivially ask for features to be written by one model and verified by another. Have Opus write it and GPT or Gemini verify it.

I figured they were entirely separate models and therefore unlikely to hallucinate in the same way, so it gave me a quick sense of confidence.

Currently I use claude code (different models but all variants of the same) so I have them do planning, review of the plan, implementation, review of the implementation, and unit tests. It's fine, but copilot felt easier.

by Kon5ole

This is the dude who ushered in the age of nonsensical boilerplate-ridden OOP code where you need to go down a bajillion of abstraction layers to see the actually implementation. Makes sense that he doesn't care about the bajillion lines of code AI produces as long as it looks good (on the surface).

by anvuong

Ironically from my experience seeing AI code a lot, it probably follows his clean code verbosity beautifully

by rpunkfu

I keep posting this but it keeps being relevant. I had an agent implement a feature completely backwards. It wrote a whole bunch of tests proving the correctness of the implementation. All the tests passed.

The really interesting thing to me is that formal verification wouldn't have helped there either -- it would have just written a mathematical proof of the correctness of the backwards feature.

by andai

This is the guy that thinks optionals are too complicated? https://blog.cleancoder.com/uncle-bob/2017/01/11/TheDarkPath...

by daaaaaaan

The author of clean code (who makes his living consulting on how to write software) has pivoted to AI in a spectacular tweet about how to set up automated software development just in time to make his living consulting about how to set up automated software development.

by jvanderbot

Join the discussion

Write your take first — we'll ask for email only when you're ready to publish.

Hacker News
Right now there is about a 95% chance that when I manually review an agent's code I find nothing that needs changing.
I find it plausible that an extra agentic review pass and more testing can bring this number up to the point that one never needs to review code again. AI writes pretty good code nowadays.
(You still need to be diligent and decide the architecture during the planning, and read the gotchas and "things to note" that the agent will spit out at the end of implementation if it had to diverge from the plan.)
by 2001zhaozhao
My current code base has more tests than runtime code. This sounds far more thorough than that. If you limit the possible productivity improvement to a small portion of total effort, it's inherently also going to be a small improvement.
Which, is fine, but not what most people are hoping for with these tools.
by nitwit005
>My current strategy is to not read any of the code written by my agents. That’s the only way I can take advantage of their productivity.
When you start getting good results from agents you soon realize you are the bottleneck.
Automating the verification of the code is the way to go, otherwise it's just not worth it. It takes longer to read and understand code than to write code, so why bother with agents if you are going to manually review it all anyway?
One thing I miss after ditching Copilot (it got too expensive) was how I could trivially ask for features to be written by one model and verified by another. Have Opus write it and GPT or Gemini verify it.
I figured they were entirely separate models and therefore unlikely to hallucinate in the same way, so it gave me a quick sense of confidence.
Currently I use claude code (different models but all variants of the same) so I have them do planning, review of the plan, implementation, review of the implementation, and unit tests. It's fine, but copilot felt easier.
by Kon5ole
This is the dude who ushered in the age of nonsensical boilerplate-ridden OOP code where you need to go down a bajillion of abstraction layers to see the actually implementation. Makes sense that he doesn't care about the bajillion lines of code AI produces as long as it looks good (on the surface).
by anvuong
Ironically from my experience seeing AI code a lot, it probably follows his clean code verbosity beautifully
by rpunkfu
I keep posting this but it keeps being relevant. I had an agent implement a feature completely backwards. It wrote a whole bunch of tests proving the correctness of the implementation. All the tests passed.
The really interesting thing to me is that formal verification wouldn't have helped there either -- it would have just written a mathematical proof of the correctness of the backwards feature.
by andai
This is the guy that thinks optionals are too complicated? https://blog.cleancoder.com/uncle-bob/2017/01/11/TheDarkPath...
by daaaaaaan
The author of clean code (who makes his living consulting on how to write software) has pivoted to AI in a spectacular tweet about how to set up automated software development just in time to make his living consulting about how to set up automated software development.
by jvanderbot