Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- "This particular report arrived privately on a Slack channel via Jane Street last week, and was itself found via Claude Fable. That compresses all timelines considerably..."
So Fable knew about it. Maybe someone is running experiments again like in OpenAI's Huggingface hack.
Cute to see that the Glasswing apparatchiks still protect their income stream and hand out no accesse.
by 29349175 - Yeah, but what does need to change? Replacing the binary at runtime without downtime is indeed one way. What about making stuff read-only on a much more expansive scale? Config, data, code, etc is all seen as the same thing and cannot be easily managed separately. Roll-back should also be made easy. Handwaving perhaps but I think additional granularity of permissions and control is inevitable.by MrQuincle
- Sounds like the next step is to start rumors about vulnerabilities so that someone else will find them and you have plausible deniability. "Hey, did you hear? Someone told me <competitor company> might be leaking their logs from a publicly accessible endpoint..."by saghm
- Transferring ideas in their abstract form is entirely possible now. What this blog post describes I believe even undersells the current reality. I've had success with finding crits by pasting in extremely high level or abstract concepts. I recently went from a talk abstract (3-4 sentences) to crits lol (finding that vuln type).by zingababba
- I do wonder what the hit rate is in general for Claude finding a successful exploit when prompted with a rumor that leads it to assume the bug is there.
“I’m told there’s a path traversal exploit in this package. Can you find it?” - probably a reasonably high chance of it finding one, even if you just made that rumor up.
by jameshart - That's basically the approach Anthropic takes.by a2ff6eeb0
- Sadly, it seems that the lesson from this, is to keep your repos private.
I am not a fan of that, but I think many folks will take that away from this.
- I built a tool that monitors commits and tries to detect silent bug fixes. With GPT-5.5-class models, it can identify fixes hidden within otherwise routine commits quite reliably. Obfuscating the code changes enough to avoid detection is difficult.
I have heard of at least one project (c-lightning?) temporarily releasing a closed-source binary as a workaround until users could update safely.
by rndhouse - People are trying to secretly fix bugs?by lazyasciiart
- Have you heard of these things called LLMs are that are quite good at reversing changes in closed source binaries?by 9cb14c1ec0
- I think roll-out and deployment are even bigger issues. Who updates their software stack within 10 minutes? Most CI runs take longer to verify the business logic is still working.
Add to that the danger of supply-chain attacks where you don't even want automatic updates.
by stephbook - Your last point is the conundrum I keep running up against
It’s a trap regardless:
A) run a known vuln B) accept and run any and all updates immediately… which could be compromised
Maybe A is worse because it’s a known vuln?
by talon8635 - I don't think this is new with LLMs (finding an exploit based on a few words offhand has always been a fun part of exploit development), but it's scaled and democratized to mass exploitation of low value targets. Backing exploit PoCs out of patches, commit messages, and random overheard or over-read sentences is a practice as old as vulnerability research. The difference with LLMs is that an explosion in actors "skilled enough" (human or not) has enabled sloppy / low-skill "exploit the whole Internet" actors in a way they weren't previously enabled.
I do agree with the author's ideas, though; most of these are things that should have been done much sooner, and I suppose it's good in a sense that there is a forcing factor now.
by bri3d - > [...] enabled sloppy / low-skill "exploit the whole Internet" actors [...]
Is there a modern equivalent for AI enabled script kiddies?
slop kiddies?
by DoctorOetker - > Backing exploit PoCs out of patches, commit messages, and random overheard or over-read sentences is a practice as old as vulnerability research
True, but it used to take days or weeks of research, testing, and RE to get those PoCs.
Today the entire chain - reading commits, RE patch binaries, building exploit, scripting exploit scan, $profit - can be fully automated and happen in minutes or hours.
by happyopossum - > The big engineering shops (like Google) have been building microupdates directly into their software to ensure that fixes directly reach users as a priority over (e.g.) being fixed in the Chrome code repository.
Yes, no. Do not even think about doing that. You cannot, should not, and must not "microupdate" code running on users machines without their consent.
Also, the whole idea is completely insane. "We might have unpatched 0days in the field, so to mitigate that, we shall add arbitrary and remote code execution via the cloud"
___
And even if you were to think that taking a page out of googles book - you know, the company known for being laser focused on having the wrongest opinions - is a good idea, you shouldn't do so for free.
Live patching is something enterprises pay a lot of money for. Making it a Foss expectation is insane.
by hypfer - It's easier to find bugs, fix them, yet there's less will than ever. My bosses just want speed and will give me a 30 minute lecture on why I don't need to solve a bug that Claude solved in 5 minutes, I've verified, and it's already in an open PR. All the while we're pushing out bugs faster and faster.
No matter how good AI gets at fixing bugs we'll never fix them when there's no will to fix things. Software will never be good if there's no will to make good software. The problem has always been about will. To many better products. It's insane that in a time where we can do better on speed and quality we still choose speed and tell ourselves it's velocity
by godelski - If your boss asks for X feature and you split it into 4 PRs, 3 that implement X and a 4th that fixes a security bug, how would they know? Someone that uses those 30mins like that isn't reviewing your PRs themselves. Just fix it and be done with it. You only needed to argue with the boss when you needed to make time for it which would delay something else. If it just appears done they'll just react with clap emoji later when you announced you also fixed this extra thing.by vasco
- My take is that when the bubble bursts good software will remain. Right now all the big players are hiding the cost. Anthropic (in my mind) is especially egregious at tokenmaxxing without telling you whats going on. A simple request to edit a text file launches multiple agents which takes ages, and burns tokens. All the while it calles it "Sautéeing" or whatever random verb they spew out as you wait for your answer.
- My boss is a big yapper too
Very low signal information, preemptively trying to cover every rebuttal despite nobody ever planning on making one, in the few times someone does he plays devils advocate endlessly
Like bro just let us babysit these agents, everything’s going to happen
by yieldcrv - Most places I've worked have been infuriatingly uninterested in fixing bugs, and would release software with major known bugs and a vague plan to fix them later. Of course, when "later" came, there were more features to cram. No time to fix those bugs.
I'm hoping one of the unintended side effect of it being essentially free to find and exploit (and fix) software bugs is that companies become less cavalier about shipping bugs in their software. Unlike most of the industry I don't believe "bugs are inevitable." Bugs are a choice developers make when they're rushing and careless and when all of their incentives are to ship quickly. You can ship bug-free software but it takes (or used to take) a really long time and a lot of care, care that commercial software developers just don't ever seem to muster.
Maybe when their software is getting 0wned over and over and 30 security issues are published a day, they'll start caring and taking their time.
by ryandrake - I guess I often don't know how good I have it as the owner of my own tech company. We've been busy fixing bugs and tackling tech debt, and I couldn't be happier.by 9cb14c1ec0
- Something that might come out of all of this is that companies that previously slow rolled security fixes will now be forced to fix them quickly, because the speed of AI and the liability of being insecure.by elmer2
- I spent about a decade coding, then half a decade in QA. QA has always been a thorn in everyone's side. The amount of money some companies spend on these departments just to ignore them is wild.
Now we've got an ability to push code out faster than ever, and absolutely zero innovation for QA. You simply cannot trust AI to verify your code is working. You can't have Quality Assurance without some form of assurance. So you're either hiring twice as many QA guys for the 10x output, or you're mostly ignoring the idea.
Economics has bubbles. Does computer science? Anyways, screw this. I'm moving to nursing.
by kulahan - It is not "will" it always is money.
There was an article some days ago where someone was claiming "with AI only you decide how many bugs you have" — well no the same forces apply because tokens are not free and business wants to do stuff that directly earns money.
Well we have to make cases and measure where the bug costs money and how. Lots of bugs are irrelevant and not blocking people from using the software. If bug doesn't drop database but is "dropdown doesn't exactly align" or "given precodnitions A,B and C something bad will happen" while A, B and C have very small possibility of occuring.
by ozim - This describes my life as an open source maintainer at the moment!
In the first 10 years of the rclone project we received about 20 security disclosures through GitHub. We had to deal with over 40 in the last month! That has taken a huge amount of my time, even using AI tools to triage and come up with fixes for review.
The hit rate for those security disclosures is pretty good - about 75% of them have a nugget of something which needs looking at. The configurations for rclone have got increasingly unlikely so I'm hoping they will dry up eventually.
I was considering just merging the fixes straight to master just to make my life easier rather than holding a dozen independent security fixes on branches and merging them at the point release and hoping not to have too many conflicts to fix up. I've decided to stick with the process for the moment.
GitHub assigns CVEs for the advisories. Before the AI apocalypse they took 2-3 days for an assignment but now it they are running at 3-4 weeks so I have to send the point releases out with CVE-PENDING in the changelog which isn't ideal.
Not sure what the solution is, but it is definitely a problem for us.
by nickcw - As long as you are not running a paid bounty program. Otherwise now you are getting 40 per day.by Kubuxu
- I feel like someone somewhere is working on an opensource GPU compute resource pool you guys could pull from as part of a donation type framework. I got a bunch of local gpu resources just relaxing, and if I could load up a a binary to provide compute for X hours a day overnight or whatever, that'd be cool.by cyanydeez
- Same with some of the projects I maintain. The reports came in for one project over a few months and then once those dried up, and they did become increasingly unlikely or appear to be edge cases, the reports started to accumulate for another project. Great to get these issues addressed but it's exhausting.by thedonncha
- Thank you so much for maintaining rclone! That project is literally the only thing that makes Google Drive an acceptable experience on Linux.