Birbla

Join the discussion

Write your take first — we'll ask for email only when you're ready to publish.

Sign in
  • Hacker News
  • I would normally suspect musl's thread stack size. Is the kernel bug confirmed?
    by walk12111
  • Interesting related article: https://www.theregister.com/ai-and-ml/2026/07/29/closed-mode...

    Looks like the AI models either refused to work or produced obvious garbage. "GLM-5.2 is what finished the job for me, re-auditing K3's work and putting together an airtight case."

    by nyc_pizzadev
  • So, why the bug triggers only with muslc and not other libc ?
    by sligor
  • Coincidence. It also only triggers on one machine.
    by inigyou
  • Probably because musl's allocator exposes single newly faulted pages directly to the app. Other allocators tend to pre-allocate multiple pages at a time, so that the race window is narrower.
    by cyberax
  • Anyone running ripgrep on a an HPC cluster against a large cluster filesystem needs to stop and redesign their workflow. This generates high amounts of small I/O which is the Achilles heel of any large cluster filesystem. You are exporting your workload onto the metadata mechanisms of the filesystem rather keeping it within the higher bandwidth capable memory subsystem on your cluster. It doesn't take but a couple users running these types of jobs simultaneously to bring a high-bandwidth filesystem to its knees. Just stop it already.
    by dosman33
  • Similarly, I've often wondered if this is the root cause of why GitHub has become fragile of late. All of a sudden you have operations on billions of tiny files that are amplified by AI usage. And object graphs are so fragmented by nature that it's hard to e.g. prefetch a page into memory and ensure that common git operations mainly touch that page's set of objects.

    And if https://isolveproblems.substack.com/p/how-microsoft-vaporize... is to be believed even in the slightest, any unoptimized paths in Azure's filesystem abstractions could cause usage spikes to have massive splash radius.

    by btown
  • This isn't an HPC cluster. It's just btrfs on my workstation.
    by dfranke
  • The analysis of the kernel bug may be a better thing to link to: https://github.com/dfoxfranke/ripgrep-3494-analysis.
    by hyperpape
  • soulless unreadable AI slop
    by yawndex
  • Not really. It's typical rambling LLM slop-analysis. It may be a correct analysis (I couldn't stomach reading it in detail), but it's a pain in the ass to read. A human doing the same analysis would have written something 1/5th the length.
    by kelnos
  • Nice sleuthing, nonsense explanation. An extra TLB flush is never an error. (The CPU is free to flush whenever it feels like doing so.) The error seems to be that somehow a zero-page PTE was present when it shouldn’t have been.

    This sounds to me like either (a) a complex race involving a CPU migration at an awkward time or (b) a bug in the zap path transiently exposing wrong PTEs.

    Also, I don’t think the zero page has pfn zero.

    If I had to throw a dart, I would guess that direct page table zapping is allowing a CPU to read through a higher-level-paging-structure cached entry to a table that has been freed and reused. Yuck. I’ve debugged one of these before.

    by amluto
  • Reading an AI-generated bug report is awful.
    by p0nce

  •     The overflow would still overflow; the use-after-free would still use after free; a musl mask race would still race.
    Hilarious. Apart from the computer poetry, the conclusion seems to be “it’s something in Linux 7.0 + musl 1.2.5”, although the only reproduction is still on the same physical Threadripper CPU and only sometimes when heavily exercised, so it hasn’t really ruled out a hardware issue.
    by fwlr
  • I tried reading and gave up at the "Headline"...

    Quoting from the bug analysis:

    >Headline. The crash is real and reproducible. With musl instrumentation we pin the in-process mechanism precisely: a thread's own store to a freshly- faulted anonymous page becomes invisible to that same thread's reload ~10 instructions later, because the page's backing is replaced mid-function. A pagemap read at the instant of the fault shows the backing is the kernel's zero page. A captured core dump confirms the crash site with matched virtual addresses. The mechanism localizes to the interaction between the per-VMA-lock anonymous-fault fast path and a concurrent munmap's TLB shootdown. A source-level review of Linux 7.0.12 identifies a specific race in that interaction, and a git comparison across v6.19/v7.0/v7.1/mainline identifies the v7.0-introduced change on the munmap-teardown side that widens it.

    "a thread's own store to a freshly- faulted anonymous page becomes invisible to that same thread's reload ~10 instructions later, because the page's backing is replaced mid-function." -> ??? What's a "backing" of a page? Freshly-faulted? Fresh fries? "~10 instructions later"?

    "A pagemap read at the instant of the fault shows the backing is the kernel's zero page." -> Backing?

    "The mechanism localizes to the interaction between the per-VMA-lock anonymous-fault fast path and a concurrent munmap's TLB shootdown." -> How can a mechanism "localize"?

    And more.

    This is... words strung together. Nothing more. I wonder how people read and make sense of this.

    In case people have forgotten what real technical writing looks like, here's a sample (I am not the author): https://yifan.lu/2019/01/11/the-first-f00d-exploit/

    by smukherjee19
  • I get why people don't bother replacing the default allocator from musl all the time (it's there, convenient). But in an application whose purpose is to be FAST, I find it weird they haven't bothered replacing it with another more performant one.

    mallocng is bad at dealing with contention during multithreading. I've had applications that usually were I/O bound suddenly become "malloc" bound when building with musl in multithreaded scenarios (and only just 8 threads). Switching to mimalloc improved performance by 20x, very close to what glibc offers by default, and just a bit under a glibc + mimalloc configuration.

    I get that there's a real issue there and it's interesting (to some) to address it, but it should have never surfaced this way in the first place.

    by Orphis
  • this use of musl's mallocng actually lead to the discovery of a kernel bug, thanks to its hardening. without it, this might have gone unnoticed for months, silently corrupting memory in the meantime.
    by potus_kushner
  • We use musl+mimalloc by default for our entire production operating systems: https://stagex.tools
    by lrvick
  • There is a real tradeoff:

    - The musl allocator is only slow with multi-threading.

    - Almost all other allocators have trouble reclaiming memory when using multi-threading. This often results in multiples more RSS than single threaded or musl's allocator.

    Agree with you on mimalloc. It can even be configured to be aggressive in memory reclaim at the cost of performance.

    by searealist
  • The way most programs achieve being fast is by re-using allocations. You don't need a fast allocator if you don't allocate. Nothing of what ripgrep does inherently requires frequent allocations.
    by mort96
  • It’s a kernel bug. While I agree libc allocators suck for no good reason, it seems like this work of equally likely hit other application code including mimalloc and glibc.
    by vlovich123
  • If you look at the sigsegv stack, the allocation comes from opendir which is in musl libc as well. The allocation override mechanism used in Rust doesn’t replace the allocator process-wide; it merely replaces the allocator Rust code talks to.

    It does seem like ripgrep should probably avoid using opendir from libc if it allocates using an allocator with a global lock though.

    by zeuxcg
  • > I get why people don't bother replacing the default allocator from musl all the time (it's there, convenient).

    ripgrep actually sets jemalloc as global allocator when built for 64b musl: https://github.com/BurntSushi/ripgrep/blob/435f59fc4b43af3ab...

    by masklinn
  • Heh, from the kernel patch: https://lore.kernel.org/all/CALCETrXbj__SFQMzPZhES5y6-sh4np-...

    > I saw a fun bug report in ripgrep and a studious but pretty bad AI-generated analysis

    Referring to https://github.com/dfoxfranke/ripgrep-3494-analysis which I indeed thought "that's an awful lot written to have been written by a human."

    Looks like that thread is from...today!

    by ndesaulniers
  • Until ~2000, "cellphone user in public" equated with "smug asshole." We're at that icky rejection stage with AI.

    Two years ago, that writeup would have been viewed as a generous gift of time to the community. Now we can't be bothered to read through it because we know where it came from, and that it's just worth $0.06 in tokens. Another reason (I think) is because we know what it portends.

    In a few years, digging manually through bugs will probably be a last resort. We'll go from sneering to yawning at the reports, as another AI agent reads them and verifies the fix. Just like we ignore compiler-generated assembly code and people on their cellphones.

    by gruntled-worker
  • Thanks for the fun read.
    by vaughands
  • That was a tough read, but in all the verbiage I do not see it identifying the same code or area as the real genuine human on lore.kernel.

    For those who can understand Claude better than me: did it anywhere actually identify the issue?

    by vintagedave
  • Patch was applied yesterday by Linus: https://lore.kernel.org/all/20260804003708.49830-1-luto@kern...
    by tonfa
  • Looking at the function the addr value should always equal end but also it looks like on x86 this value is discarded in pte_free_tlb so maybe this is not the problem or I’m looking at a different version.

    https://github.com/torvalds/linux/blob/master/mm/memory.c#L1...

    Assuming addr is a virtual address it probably makes sense that pte_free_tlb does not need it.

    https://github.com/torvalds/linux/blob/master/arch/x86/mm/pg...

    by benmmurphy