Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- I was using Copilot Code Review pretty religiously for a while, as I get access for free (the $10 plan) due to my Open Source work, but it recently introduced a monster of a misfeature that caused a massive increase in complexity over time, while I wasn't paying close enough attention to it. Every subsequent model saw that change and the explanation for it in the changelog and assumed it was policy rather than a model being brain-damaged, so it became a fractal of fuckery I had to untangle with a good model and some close human supervision. It was an administrative tool that directly edits configuration files for a service. Copilot code review decided it needed to be an overlay service that applied only the configuration created by our config UI, fully independent of the system service. And, subsequent "bugs" that other LLMs "fixed" were just applying more and more bandaids to that bad decision (attaching the services together so restarting one would restart the other after, etc., making sure there were no conflicts across the files, warnings when one rule conflicted with another, etc.). Because modifying an additional service is simply not what the tool was designed for, it seemed to be really buggy, so there were lots of "fixes". It took me too long to realize the fundamental failure point.
That's my way of saying, I am hesitant to trust a stupid model to do code review because I become complacent and when it suggests a small change that seems reasonable (the LLMs are very good at sounding reasonable, far better at sounding reasonable than being reasonable, in fact), I might not notice that it just did a stupid until much later, when it becomes a big pile of stupids.
My fault for trusting it, of course. But, my eyes glaze over when I read AI prose, whether it's code review or anything else. It's hard to catch one incorrect behavior in a batch of several reasonable suggestions.
by SwellJoe - False positives have a real cost, especially if AI is reading a review. Consider if you have GPT-6 Astra looking at a review and finding a bunch of false positives it burns tokens to figure out.by rektomatic
- They state Luna is good enough, but its accuracy of findings is 74% whereas Astra is 96%. Dealing with false positives is expensive.
I am finding AI doing its own reviews as part of the process to be the key to productivity. I do subagent (fresh context reviews) at multiple stages with well-specified review criteria. It is really expensive to do with OpenAI or Claude API billing. Deepseek or the discounted monthly plans from OpenAI or Claude can be discounted similar to the 28x they state for Luna compared to Astra and you maintain much higher quality.
by gregwebs - I found Luna and even 5.4-mini to be quite good at code review provided a few things:
1. Run it in multiple cycles, only on the diff, and only emit a few findings at a time.
2. Give it a memory so each cycle, it knows the previous finding to check if it's been fixed.
3. Give it access to canonical docs that encode your human reviewer heuristics. I exposed these as tool calls so they could be tracked via telemetry.
4. Run multiple reviewers, each with a tight focus. Security, performance, structural, database, etc. Each a separate prompt and persona. Additionally, we had file activation filters so the FE React reviewer didn't activate on BE only changes.
Luna and 5.4-mini with no reasoning were exceptionally fast and almost always found issues with code produced by Opus and Fable.
Default prompts for the curious (these are templates deployed by default, but customizable).
Performance: https://github.com/zeeq-ai/zeeq-app/blob/main/src/backend/Ze...
Structural: https://github.com/zeeq-ai/zeeq-app/blob/main/src/backend/Ze...
(Keep in mind each agent also has tools to access and reference external docs.)
- $0.10 extra per pr review is nothing. What software company is willing to accept worse reviews and less bugs found to save 10 cents?
- I only use chinese models for code reviews because you can actually tell them to take an adversarial stance and actively look for security issues without risking refusals. GLM-5.3 has been great for this, although it can be slow on larger PRs.
- IMHO, Codex with Astra/Sol and Claude with Fable/Opus are all any professional programmer should be using in Sep 2026, if they can afford it.
These models are still terrible compared to what we'd actually wish for, but they're the best available.
If you can get away with using the $200/mo subscriptions, it's really not even a money thing for most professionals.
Almost all of my work is now plan, generate, review, plan, generate, review, commit, push.
I'm using Claude or Codex (or both), and they're doing all of the testing "inline" rather than through a CI action, etc.
by jacobgold - AI should be used for code review but not in CI.
You should already have 2+ developers looking at most PRs. And these developers should absolutely use AI. The PR author should use AI.
But what you should not do is pipe the AI output directly into the PR and tell the PR author to deal with it. That's adding noise to the PR review process. Everything it says is something the PR author needs to validate as relevant, helpful, etc. A human needs to do that before confronting the author with it.
You wouldn't ask an agent to review a PR then just copy/paste the output into the PR, would you?
by nonethewiser