Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- This is super fascinating, and I loved seeing testing of where exactly you need frontier intelligence -- looks like coordination / planning, but not coding right now -- the article's a bit of a tease, as we can't play with such a harness, or their new version control system or get a workable artifact out of it.
That said, I love the work on figuring out these harness coordination jobs. While there are analogs to human management there is also this enticing feeling that, since the models are broadly deterministic, we might be able to get repeatable science-type lessons about managing them with enough testing.
by vessenes - I initially thought that getting agents to work for longer and in large groups was the future, but I'm increasingly thinking that, at least for engineering, just one thread makes more sense. The agent pulls things into context as needed. One thing that I've been experimenting with is also letting the agent remove things from context, such as files. But just adding to the context and compacting when it's full seems like it might beat a lot of more advanced options. Because the model is good, it knows what to put in the summary; just enough for the model to be able to rebuild the context from that seed (e.g. pulling in relevant files/data into the context).by edg5000
- Have you used this method yourself for long workflows and with contexts approaching 1 million tokens? It doesn't work very well. LLM context is nearly half unusable.by Schlagbohrer
- Even at a much smaller scale of a few dozen parallel agents working I've found real benefit to a structured hierarchy of agents. Not just keeping the context clean and focused at the right level, but if you find that something has just gone off the rails, you can more easily rip out or even just fully delete a leg of work without damaging the original design. It's also nice when you find a design change is needed and you can communicate that to the top level and have it efficiently propagate down where it matters.
I like it so much I even recorded a video walking through the process!
https://www.youtube.com/watch?v=efUmcCiRoDU (I do walk through how I do this in our app DevSwarm here)
by trevswarm - Today it is SQLite, tomorrow Linux?
I'm not sure of the long-term viability of these brute force swarm approaches, but clearly the AI companies dig the marketing they get out of it. But the fact that they get so far at this stage suggests we have a very interesting 2 years coming up.
- > One reading is that it was more productive. Another is that most of those commits were busywork (thrash, contention, churn).
Kudos
by timcobb - This is almost a year behind Steve Yegge's first post on beads. Gas Town and Gas City provide orchestration for the swarm. So far I haven't seen a perfect implementation but this idea isn't new.by smoyer
- Does every peice of prose published here have to be novel? is it not enough for you to read a new. perspective?by nullsanity
- Everything Steve yegge has done has been trash. That's why nobody is talking about beads or gas town. It was clear even in the beginning that it was a borderline AI-psychosis-fueled trash fire.by IceDane
- I don't think Gas Town was actually all that much ahead of the curve - see Cursor's swarm post from January this year, mentioned at the very top of this blog post.
I also don't know that beads is a very strong example that Yegge knows what he's doing - it's famously been described as (pseudo)malware.
by sudb - Am I reading this right? Opus + Composer did a comparable job to Fable, at ~1/19th the price, and half the LoC?by dctwin
- I'm struggling to use Fable correctly. Everything I do ends up getting flagged and reverted back to Opus. Need to do some digging.by LostMyLogin
- Point is, frontier models don't necessarily are better than a swarm of non-frontier ones for these scoped problems. IMO frotnier models excel at underspec'd or more complex problems where ideation and exploration are key (and that's where they become crazy expensive).by espetro
- Yea, looks like it (at least for their v2). Fable did much better in V1, but once they added their tooling around it, Opus + Composer ended up doing better (>0.8 Grade on the SQLite chart) for the final product. Granted, Fable reached a 'good' result (0.7~) in a much faster timeframe.by KyleTheDev
- I would have loved to see more of the harness engineering shared as code. Instead we are left with only the outcome.
I guess that makes sense since the harness is the product in the case of Cursor.
by whinvik - I don't think there is a lot test harness, given there is a golden implementation (sqlite3), with clear interface (SQL, and sqlite3 db files). Once you slop crappy implementation, then you can create one group of agents finding gap where slop doing wrong vs golden, and another group of agents filling the gap meeting the conformance.
I assume they did similar, as long as test case(or spec) can be parallelized. Having parallel execution / testing environment is crucial if we need 1000 commits/second throughput.
I would not surprised they have 1+ millions of specs.
I would bet they won't release generated specs - actually this will be a valuable asset like https://sqlite.org/th3.html
by blmarket - I call this junk "meta-agentic engineering", it reminds me of people who have the coolest nvim configs, spend hundreds of hours customizing it but ultimately get less work done than the guy with minimal workflows, if any at all.
I look on twitter and it's just people building tools for agents to use agents, some weird customization loop going on in the LLM space right now. Ultimately these are trends pushed on us from model providers because they 10x token consumption. Its literally just BS trends to increase revenue at these companies, most of it is largely useless.
by dakolli - How do we know if these models weren’t trained on Turso’s rewrite of SQLite in Rust?
It seems both likely that they were and impossible to remove that code from pretraining. Doesn’t that make this just about LLM memorization of the training set? What am I missing?
by shay_ker - Wouldn't that be in the authors' interest to disclose, if true?
- This is more about automated long horizon work.
- > What am I missing?
They’re testing the same models on the same task but different ways of organising the swarms and the new approach works better.
by IanCal - > To test that progress, we returned to a task the old swarm had struggled with: building SQLite from scratch, in Rust, from nothing but its documentation.
Isn't SQLite's source code in its training data?
- not in Rust. might be different enough ...by Romario77
- Of course it isby overgard
- >The browser swarm from earlier this year peaked at roughly 1,000 commits per hour on Git. The new system peaks at around 1,000 commits per second.
>To facilitate this rate of activity, we built a new version control system (VCS) from scratch. Throughput was not the only reason to own this layer. Every change in the system passes through the VCS, so it is where collisions first become visible, and several of the coordination mechanisms in the next section are implemented directly inside of it.
Talk about inventing the universe to make a button.
by htrp - I mean why even use VCS at that point?by leoqa
- What would have happen if you do not commit at all?by h2aichat
- They probably just vibe coded it with ChatGPTby Schlagbohrer
- Seems very reasonable to me tbh, the VCS was built to facilitate human work structures, rebuilding it to facilitate autonomous work seems appropriate.
- It's hard not read such quotes and immediately think of the Infinite Monkey Theorem.by jgalt212