Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- i think gpt 5.6 luna-max in particular is excellent for non long term agentic use
code reviews, unit tests, docs, whatever you dont want more expensive models working
i think the new muse contributor model is enticing too if you are not using it for private/sensitive stuff
by zergrush - This article is missing an incredibly important detail: what is the harness doing?
I get remarkably good results using any recent OpenAI model using the codex-rs harness pointing at a built checkout of the PR. The models use the available tools (i.e. the shell) to understand the repo. I get some false positives and some false negatives, but I don’t believe for a second that I would get comparable performance using a dramatically less capable harness. (Also, the models read the short AGENTS.md for some context as to what’s going on. The prompt I use is about one sentence. I don’t bother with the built in review tool.)
by amluto - Updated link: https://entelligence.ai/blogs/gpt-6-astra-cost-1.6x-more-per...
The current link is a 404, seems like they didn't redirect it properly.
by AntonyGarand - Use cheap models if your code is low value.by xnx
- I was using Copilot Code Review pretty religiously for a while, as I get access for free (the $10 plan) due to my Open Source work, but it recently introduced a monster of a misfeature that caused a massive increase in complexity over time, while I wasn't paying close enough attention to it. Every subsequent model saw that change and the explanation for it in the changelog and assumed it was policy rather than a model being brain-damaged, so it became a fractal of fuckery I had to untangle with a good model and some close human supervision. It was an administrative tool that directly edits configuration files for a service. Copilot code review decided it needed to be an overlay service that applied only the configuration created by our config UI, fully independent of the system service. And, subsequent "bugs" that other LLMs "fixed" were just applying more and more bandaids to that bad decision (attaching the services together so restarting one would restart the other after, etc., making sure there were no conflicts across the files, warnings when one rule conflicted with another, etc.). Because modifying an additional service is simply not what the tool was designed for, it seemed to be really buggy, so there were lots of "fixes". It took me too long to realize the fundamental failure point.
That's my way of saying, I am hesitant to trust a stupid model to do code review because I become complacent and when it suggests a small change that seems reasonable (the LLMs are very good at sounding reasonable, far better at sounding reasonable than being reasonable, in fact), I might not notice that it just did a stupid until much later, when it becomes a big pile of stupids.
My fault for trusting it, of course. But, my eyes glaze over when I read AI prose, whether it's code review or anything else. It's hard to catch one incorrect behavior in a batch of several reasonable suggestions.
by SwellJoe - The "assumed it was a policy" thing resonates with me. My agents have such trouble distinguishing a note from a law, and they seem to looooove following laws. I've had similar fuckups; a stray constraint can turn into some wild tradeoff decisions which compound if you aren't paying attention.by threecheese
- False positives have a real cost, especially if AI is reading a review. Consider if you have GPT-6 Astra looking at a review and finding a bunch of false positives it burns tokens to figure out.by rektomatic
- They state Luna is good enough, but its accuracy of findings is 74% whereas Astra is 96%. Dealing with false positives is expensive.
I am finding AI doing its own reviews as part of the process to be the key to productivity. I do subagent (fresh context reviews) at multiple stages with well-specified review criteria. It is really expensive to do with OpenAI or Claude API billing. Deepseek or the discounted monthly plans from OpenAI or Claude can be discounted similar to the 28x they state for Luna compared to Astra and you maintain much higher quality.
by gregwebs - My thoughts exactly. That 22pp gap is massive, and it seems to me Astra is worth every extra penny.by samusiam
- I found Luna and even 5.4-mini to be quite good at code review provided a few things:
1. Run it in multiple cycles, only on the diff, and only emit a few findings at a time.
2. Give it a memory so each cycle, it knows the previous finding to check if it's been fixed.
3. Give it access to canonical docs that encode your human reviewer heuristics. I exposed these as tool calls so they could be tracked via telemetry.
4. Run multiple reviewers, each with a tight focus. Security, performance, structural, database, etc. Each a separate prompt and persona. Additionally, we had file activation filters so the FE React reviewer didn't activate on BE only changes.
Luna and 5.4-mini with no reasoning were exceptionally fast and almost always found issues with code produced by Opus and Fable.
Default prompts for the curious (these are templates deployed by default, but customizable).
Performance: https://github.com/zeeq-ai/zeeq-app/blob/main/src/backend/Ze...
Structural: https://github.com/zeeq-ai/zeeq-app/blob/main/src/backend/Ze...
(Keep in mind each agent also has tools to access and reference external docs.)
- Only on the diff? Why? I found AI reviews garbage until they stopped being only on the diff and were actually able to query real context.by therealdrag0
- $0.10 extra per pr review is nothing. What software company is willing to accept worse reviews and less bugs found to save 10 cents?
- Unless you're detached from reality our industry is filled with billion (trillion) dollar companies shoving broken crap on prod written by MIT-bred leetcode Ninjas and it never mattered anyway because code has no value and has always been throwaway except very rare instances.by epolanski
- Exactly this. It's still (at this time) cheaper than a developer that would most likely perform worse.by po1nt
- That very much depends on how code will be written in the future, how much of it and how often it changes. If more of it will be ephemeral (kind of what agents are already doing for all sorts of tasks right now) finding ways to very cheaply check might be of high value.
(I suspect this won't be it, though. Probably something the model providers are going to bake into the models themselves.)
by jstummbillig - Worse seems subjective here. It seems Luna found bugs Astra did not, and vice versa. Astra had lower noise overall. I think my take away here is to use a blend of models given their different abilities to find different domains of bugs.by mooman219
- I only use chinese models for code reviews because you can actually tell them to take an adversarial stance and actively look for security issues without risking refusals. GLM-5.3 has been great for this, although it can be slow on larger PRs.
- IMHO, Codex with Astra/Sol and Claude with Fable/Opus are all any professional programmer should be using in Sep 2026, if they can afford it.
These models are still terrible compared to what we'd actually wish for, but they're the best available.
If you can get away with using the $200/mo subscriptions, it's really not even a money thing for most professionals.
Almost all of my work is now plan, generate, review, plan, generate, review, commit, push.
I'm using Claude or Codex (or both), and they're doing all of the testing "inline" rather than through a CI action, etc.
by jacobgold - I disagree; I just spent 15x dogfooding some Claude setup I rolled out to the org making changes that would have cost me less then a dollar had I used Luna and I would have got the same, if not better results; better because it would have been faster so I could have iterated more.by ghthor
- Could you elaborate on your exact setup? Where do you run these models?by baxtr
- Qwen3.8 is all you need.by cyanydeez
- I just tell everyone to use Fable 5.1 for everything at this point. Astra is unfortunately a dud, I'm sure they will try to fix a bunch of it with GPT-6.1 but OAI has had this issue for awhile now where every other generation has some sort of strange tic, or reward hacking issue, or something. It's almost like they are balancing the RL on the tip of a needle.
Opus 5 has issues too, comment-slop, claude-ish, etc.
5.1 on the other hand can seemingly do no wrong. Easy to work with, writes human-level code. Expensive, yes, but even at Low effort it's well worth it.
by extr - With everything changing it's great to hear others have the same workflow. I added a snapshot step so I'm doing
plan, generate step 1, review, snapshot, generate step 2, review, snapshot...
That way I have a chance to diff with the previous iteration and clean up comments, modify skills, etc. also if it bonks on a step I'm one snapshot away from trying again...
Is there a place people share their workflows other than HN comments?
- Unfortunately ChatGPT just stopped allowing people to upgrade to the $200/mo subscription.
I started my first paid subscription ($100/mo) last week, and now I want to upgrade and I can't :-(
by rationalist - For side projects I pretty much exclusively use Luna xhigh. The $20/mo plan with the recent generous resets is more than enough for me. Sometimes I reach the 5hr limit, but haven't reached the weekly limit yet.
The most recent project it finished was a SIP client for an ESP32 in-wall touch panel that I got from AliExpress for $50. It rings when someone is at my doorbell and let's me answer calls and see video. Yes an ESP32 can stream H.264 video :D
My only complaint with Luna is it seems to give up when the work is half finished, and I often need to tell it to continue. But I feel this is mainly a harness problem. I just use it in ChatGPT/Codex as it gives me easy remote access to check in on what it's doing.
(At my dayjob I usually spend $200+/day with Opus/Fable)
by fy20 - AI should be used for code review but not in CI.
You should already have 2+ developers looking at most PRs. And these developers should absolutely use AI. The PR author should use AI.
But what you should not do is pipe the AI output directly into the PR and tell the PR author to deal with it. That's adding noise to the PR review process. Everything it says is something the PR author needs to validate as relevant, helpful, etc. A human needs to do that before confronting the author with it.
You wouldn't ask an agent to review a PR then just copy/paste the output into the PR, would you?
by nonethewiser - Human reviews are going away for most PRs these days. If the LLM decides the risk level of the change is low, the PR can be merged without another human in the loop. That's at least the direction many companies are taking. Only require a human judgement when necessary.by rockinghigh
- I would and do.
works perfect
https://github.com/dzmitry-lahoda/dz/tree/main/agents/skills...
burns half of day sub of astra for 200 USD. runs 1 hour on our repo.
finds bugs missed by coderabitai, devin-integration-bot, codex and copilot (I ask them first until nothing found, yet my orchestration finds more).
I do not hardcode our application in orchestration, but run subagent for applicationdomainproduct detection.
also I use agy 200usd sub for second-opinion as one of steps for false positive elimination.
by dlahoda - > You wouldn't ask an agent to review a PR then just copy/paste the output into the PR, would you?
I would if the PR was clearly written by AI. I'm fine with the PR author using AI, but only to draft the PR. They should be editing the shit out of it for the final version before submitting it.
- We had a two human PR requirement until recently we dropped it. It was slowing us down too much now the human developer creating the future is obviously writing it all with AI so they need to check it then depending on the feature and it’s use it requires a PR but it’s not universal and we’ve stepped up our automated test Tan X what it used to be it’s been so far fewer bugs better deliveryby mchusma