Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- I love hearing about this product, I hope one day I'll get to use anything like it.by LoganDark
- Omg the one thing I hate more than ai art is ai pixelart where the pixels are all different sizes
- Related: finding the same bug with TLA+ https://news.ycombinator.com/item?id=48730953by archseer
- related: Tailscale Traces Database Corruption to 16y/o SQLite WAL-Reset Bug (tailscale.com) https://news.ycombinator.com/item?id=49272832
- We wanted to publish this ASAP since people are talking about this bug today. In a follow-up, we will show how our automated causality analysis could have saved Tailscale and SQLite from 6 months of root cause analysis. Stay tuned!by wwilson
- I feel like we’re in a weird P=NP age, where after someone finds an issue, someone points out a cheaper way to find the same issue, which inevitably involves knowing the issue is there in the first place and it’s approximate location.by moomin
- I went clicking through to see if I could find the prompt they fed the AI to locate the issue / write the test suite.
I couldn't find it, so its unclear if the prompt was completely "make a test suite" or was lead towards finding it in the first place, which wouldn't be a fair test.
The closest I mention of the prompt I could find was:
> Then I asked it to write a simple workload which exercised the WAL insert and checkpoint code. Notably, this is a completely generic workload.
With a skeptical lens, unclear.
by minimaltom - Their founder guy says[1] they found some other bugs while doing this that they will report soon, so if that is true, seems more plausibly like a pretty generic thing.
Looking forward to seeing the other bugs they found.
by gamegoblin - Hey, author here! Good question. The prompt was to use the Antithesis skills[1] to write a workload that exercises SQLite's insertion and checkpoint machinery across multiple processes using a specific version of SQLite (3.51.2). The agent who did the work was aware of the bug and knew I wanted to see if Antithesis could reproduce it. However, I specifically designed the workload the same way we approach testing other WAL checkpoint-style systems: by putting them under stress. The workload itself does not know about the bug.
My goal, which I believe I accomplished, was to determine if Antithesis would have found this bug if we had put the SQLite WAL under test.
by carlsverre - I understand the author feels he missed an opportunity to catch a bug in Sqlite - but this still feels like catching something in hindsight. Just knowing the bug the is in Sqlite's concurrency engine is a huge win.
But reading over tailscale's blogpost, I'm not sure how I would connect the dots to "this issue I'm seeing in production" to "load this up in antithesis". It seemed half the battle was figuring out if the bug was in Sqlite, Tailscale, or Linux, and in that situation if you gave Antithesis all the context, would it reliably find the bug?
by nemothekid - Maybe it wouldn't, but being able to cheaply test SQLite within a few minutes is still very valuable, even if just to rule it out.by stavros
- as a long time lurker who usually enjoys antithesis posts, this was a pretty existentially sad read. we all know that company blog posts here always tend to be ads at their core, but historically afaicr antithesis generally toes the line well of keeping the "thing we sell is really good" as a side dish to the "i (author) did very interesting deep thinking and/or hard work to accomplish something cool" entree.
this one falls very flat on that front imo. the "antithesis finds rare bugs" is the overwhelming majority of the actual content here, where as the usual meat of the blog, the hard work, the deep thinking, really distills down to "typed this into claude mobile, typed that into claude mobile". yes, its a statement to the power of the technologies of today that's all it takes to find this bug, but man it's a sad signal for those of us looking for compelling technical reads instead of b2b sales pitches.
- I am okay with companies shilling their tools. Ultimately I'd want my employer to buy good tools that make our lives easier specifically wrt debugging which I genuinely hate. However this post could have been a better walk through about what Claude found, what was the exact root cause and how it can be fixed instead of the generic Anthithesis found it in 15m.
- Hi! I'm the author and the person who was on a road trip when I decided to do this experiment back in July. I'm super excited to finally get the blog post out, and even better, alongside a post describing the root cause process that went into the project. You can read about Tailscale's story here: https://tailscale.com/blog/sqlite-wal-reset-bug
I'm only sad that I didn't put SQLite under test earlier in the year, or I would have found this issue right away. If you look at the workload[1], you can see how simple it is. Exactly the same kind of workload we write every day to help our customers find bugs.
If you have any questions about our process or how debugging with Antithesis works, please let me know! Thanks for reading!
[1]: https://github.com/antithesishq/sqlite/blob/3.51.2-instrumen...
by carlsverre - Typo in the post: "SQL 3.51.2" should presumably be "SQLite 3.51.2".by cperciva
- Nice article, but could you get someone to fix the blog layout? Having the author card pinned to the window rather than being able to scroll past it is distracting.by skybrian
- So you got information where the bug is, and now you brag that your AI friend found it quickly :joy:
Go find bugs that haven't been yet found, and then brag about them :joy:
- you asserted that Antithesis would have found it right away, but it seems like from your write-up that you first set up instrumentation on the source, then told it to reproduce the WAL bug.
i don't feel like i gained a strong appreciation for what the actual Antithesis capabilities are or why you had confidence this was anything the Antithesis tools would have helped find? i want to know more about what kind of analysis or capabilities Antithesis has and how it could have helped find the issue. what's clear is that Antithesis could confirm an already found issue. and even there I don't feel like I gained any connection or knowledge or understanding of what the tooling was doing or how it would help. i'd really like more technical meat to this post to understand the value proposition.
- How difficult would it have been to isolate that problem if you didn't already know the SQLite subsystem it was in? This feels, to someone relatively ignorant of the SQLite / Tailscale / Antithesis architectures as a "hindsight is 20/20" kind of thing, but I'm open to learning more.by MPSimmons