Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- Hey, the Python thing is interesting! I’ve noticed that Fable, too, likes to write tons of Python, even for just replacing a few lines of code. Before that used to be either some kind of internal thing or regular awk sed, now it’s full python scripts.by Toutouxc
- My own observations are that I used to target turn lengths of 10-15 minutes and these new models (since 5.6) extended that a bit to ~25 minutes, as they tend to do more tests and reviews. Targeting hours-long turns makes as much sense, as putting on cruise control and going to sleep.by exitb
- They are probably using Actual Indians. If it takes 25 minutes you can just type the code yourself.by z3t4
- 51 comments so far, the vast majority panning Astra's coding abilities. An uninformed reader may come away with the impression that this isn't an absolutely revolutionary technology that with coding abilities many of us thought were not even going to be possible with language models as recently as a year ago.
Yeah, it's not perfect, but it's really good and extrapolating this rate of improvement for 6 months is rather terrifying (from a SWE perspective, at least).
by nvrmnd - It's impressive technology but not revolutionary. Revolutionary technology would have resulted in, you know, a revolution in software quality. Instead quality keeps going down.
- i think the main consensus here is that the actual performance is not indicative of the benchmark performance (which supposedly outperforms the previous iterations)
- There was a big jump around new year, but they seem to have flatlined since them. Just my experience.by sbt
- gpt-6-astra is a bitch, it constantly scope creeps itself with "yet another thing" to give it that darn polished lick. the results are eventually a little bit better but at what cost? let's do the math.
gpt-5.6-sol: 1x base gpt-6-astra 2.5x base in subscription
then gpt-6-astra tends to spawn subagents a lot, often with all kinds of models such as gpt-5.6, 5.3-codex etc., which is neat. it's a good coordinator but even more cost.
and then it tends to run _full test suites_ over an over again (each costs like 15 minutes) just to verify that _one test_ was fixed etc., and does so for as long as until the test is fixed, eventually accumulating 2 hours or so.
yesterday I assigned it a task to rebase my changs in a repo onto the latest upstream changes. while gpt-5.6-sol consistently took like an hour to do so end-to-end, astra ran for more than 6 hours and still wasn't done. it kept finding "one more thing" that was goldplating that I didn't ask for.
- have you ever worked for a big company where that's the status quo for any tiny change... hours on _full test suites_ over and over again.by djmips
- I might be missing something here, but can't you just put in AGENTS.md something like "do not run full test suite unless asked" or something?by djeastm
- They don't always have a great concept of time so for something like running a full test suite that takes a long time you should just tell it not to do that
- > each costs like 15 minutes
I've got a custom agent loop that will reuse unit testing results if no apply patch operations occurred since the last invoke.
Wall clock time isn't something I would put on the AI provider. That's entirely a consequence of the system that you've brought to the party.
by bob1029 - I've observed the same thing where the new models want to run obscene bash commands or python scripts which are completely unreadable and utilise every option flag that exists.
It's impossible to review. These commands are less readable than regex.
by Gigachad - Cursor was pretty amazing until it started going to shit. High prices, UI changes, moving MCP settings, and moving other things around.
Most importantly was the Index change. It was a very unique thing to Cursor that you could use .cursorignore to control what it sees and then index the directory. Then the built in Cursor AI harness could find code and files like magic. They have since obscured the Index feature out of sight recently and I’m not sure how it even works anymore.
This granular control not only helped with privacy, but it also helped make everything more efficient because the AI didn’t waste time and tokens looking at files that aren’t relevant.
So, now, like everyone is talking about, we have really inefficient ways of how the AI is reading files because there is no first-class approaches.
Also, agree with consensus that Astra is weird.
by iJohnDoe - If these tools are as clever as they seem then why not just tell them to rewrite the code in a more review friendly style?
I only dabble in the use of LLMs to generate code for hobby programming (I'm retired from software development) so I don't use any specialised tools.
I almost always have to tell ChatGPT (via Duck AI usually) to rewrite several times even when it has produced a workable script just because it has often used some unnecessarily roundabout way of achieving something. Usually with extra prompting I can get something that is both more efficient and more readable.
by ninalanyon - I noticed that too so I appended to Claude Code’s system prompt a reminder to use the standard read/write tools, but since Claude Code switched to default auto-mode, I’ve seen it imply that the auto-mode tooling encourages the use of bash-only commands (sed, python, etc) which has a whole slew of negative side affects.by chambored
- I'd like to submit my counterpoint. I work on an established codebase building new features and fixing bugs. It has access to our story board, git and a couple of other mcps. As long as the story is well written with clear requirements and expectations it always produces quality code that I validate as a human with a variety of tests automated and manual. I peer review the code. My colleagues then peer review that too.
I have noticed two things - new features take at a minimum at least half the time it took me previously and bugs are much less frequent. Even faster when bug fixing.
My takeaway is that you need solid requirements, clear context and thoughtful human oversight primarily during planning but also during verification
by _usefulcat - I don't think this is a counterpoint.
An established codebase is already the best kind of context you could give an agent. It has all the patterns baked in so the agent simply follows established patterns. Such a codebase probably contains tens to hundreds of thousands of man-hours poured into it by humans refining it to do what it does - taking into account real world feedback and constraints.
When working on something from scratch, the best an agent can do is the average of whatever is in its training set and the clarity of the text prompts.
by 0xpgm - Early lesson I learned from AI engineering was - there is no substitute to giving a groomed epic to an agent. Instead of simply saying 'implement themes in my product' you need to be specific, in fact more specific than usual. You need to say exactly what is in scope and what's not, even down to a buttons, events and layouts.
You can groom the epic with the help of AI, but final review must be done by someone who can take ownership of the specs and hence is responsible if something has fallen through the cracks. AI's response will be limited by the output tokens of that specific agent, and there will no repercussions for AI even if it accepts its mistakes.
by gps372 - If you define what you want in typescript by updating existing typescript wordage, you get what you want exactly as specified, with minimal token cost and no ip theft.by matt3210
- Genuinely curious, what is the benefit of doing this over just programming it yourself? It sounds to me like this is just an extra step that will grant you less control?
When programming a feature, you are simultaneously doing at least four things: 1. Implementing. 2. Building highly detailed mental models. 3. Learning and expanding your skillset. 4. Quality control and scope limiting.
And this process can be iterative and dynamic. Writing massive, super detailed specs that you then hand off to a undeterministic model feels like doing step one and three, while skipping two and four, which you then have to do after. What is the benefit? The speed up, in my opinion, comes if you skip step two and four, but then your product WILL be worse. Feels like I am going crazy?
Programming was never the bottleneck for software dev?
by civvv - But isn’t “grooming” the hardest part? To know the details means to know the codebases behind. So you do discovery, read code, ask people, check db tables, and then once you figure it out, you write a doc with the proposed solution. It gets reviewed and then you code it.
I definitely need AI help for the discovery part… so it always starts with a simple “I need to do X”
by sdevonoes - > in fact more specific than usual.
At which point you might as well write the code yourself and get a deterministic result faster, better and cheaper.
- > Instead of simply saying 'implement themes in my product' you need to be specific, in fact more specific than usual.
Around February you could get away with very vague prompts to Claude. I feel like models have regressed since
by troupo - > Early lesson I learned from AI engineering was - there is no substitute to giving a groomed epic to an agent. Instead of simply saying 'implement themes in my product' you need to be specific, in fact more specific than usual.
I think this lesson is getting partially outdated. Yes, you need to be specific about what you want, and with earlier LLMs, you need to had both domain knowledge and some general software development experience to front-load various big and small choices about design, architecture and operational reality - what libraries to use, how system components communicate, how you handle auth and store secrets, etc. Otherwise the LLM would pull some random mix of ideas from its latent space, and give you something that's broken in really stupid ways.
Nowadays, it doesn't feel like that to me, not anymore. I still need some understanding to verify the proposals, but I found the last ~6 months of SOTA models to make good choices. Like, just yesterday I asked Claude to design me some simple service, and focused on explaining it the domain parts (nature of systems I want to integrate together, the purpose of that, and the user's priorities and use cases), and the design I got back had specific suggestions around security, authentication, deployment, failover, integration, behavioral impedance-matching between integrated systems, and more, that I all recognized as based on solid software engineering and ops practices, but deviating from it explicitly in every place where it would be wrong for this specific project. The model considered way more corner cases than I did, and I'm actually really impressed by it.
But then, I find greenfield development is easy with LLMs. Modifying existing systems, especially legacy ones, is where I need to babysit and micromanage models - because any misunderstanding or inaccuracy, which often comes from stale documentation or naming mistakes, tends to get amplified and confuse the agents. No matter how precisely you specify your epic, if the model will find something that contradicts your knowledge/intent, there are good chances it'll get confused and make subtle errors, and you won't realize until much later.
The way I see it: models are highly biased to treat everything they read as "ground truth", all of equal importance. There's no nuanced notion that some information may be stale, that there's a temporal and causal order to sources, and that some information may just be wrong.
And this compounds when you let your LLM write code and documentation over time.
by TeMPOraL - Yeah, I think as a general direction we need to allow a useful collaboration with the agents on the issue board.
This is btw why Epiq was developed, to keep the board as code, git-backed, distributed (via an event log mechanism), and with the ability to replay the board, to see what agents actually did:
by samuell - I’ve observed exactly these patterns with Opus and Fable as well - for example, forgetting that they can edit files and instead use python scripts as a patching tool…by buildbot
- Sounds like you don't have enough experience with coding agents. Deterministic scripts must always be preferred instead of LLM tool calls. In fact, you should instruct your agents to write code to execute instead of letting them call tools.by d5lt5
- I use AST replacers, much more reliable.by ZeWaka
- this is intentional, afaik agents do better with python and alike than the harness tooling.