Birbla

Join the discussion

Write your take first — we'll ask for email only when you're ready to publish.

Sign in
  • Hacker News
  • A WebX technology that actually involves browsers!
    by adastra22
  • Bake me a cake

    responds with

    > Error: Cannot initialize runtime because of requested maxStorageBuffersPerShaderStage exceeds limit. requested=10, limit=9.

    by conceptme
  • What browser and hardware are you using?
    by erichdongubler
  • You might like webml-kit https://npm.im/webml-kit
    by init0
  • I love it! It works fine for me on Google Chrome on my MacBook Pro 14-inch M5 16GB RAM and 10 Cores CPU and 10 Cores GPU, with Metal 4 support
    by karuppiah7890
  • Oh nice, this actually works in Safari on my Mac! Prefill: 181.2 tok/s, Decode: 58.5 tok/s on my M4 Max -- not too impressive for 1B, but definitely impressive that it runs in my browser!
    by LoganDark
  • Is there any use case/demo of lighter models like using embeddings for full text search, STT, OCR etc?
    by aitchnyu
  • I really enjoy this engine. I’ve used it for personal projects, but it hasn’t been updated since Gemma 2. I suggest using Transformers.js instead these days.
    by MarioMan
  • It is kinda obvious, but maybe that's why it's not stated anywhere: each browser session will result in a download of 500 MB to ~1 GB, depending on your model selection. So, it's better to add a disclaimer if you end up using WebLLM in a customer-facing site.
    by mandeepj
  • Project is de facto dead, used it for many years and had to rip it out 6 months ago, don't waste your time.
    by refulgentis
  • What did you switch to?
    by skybrian
  • Yeah, both WebLLM and mlc-llm have been mostly abandoned since NVIDIA acquired OctoAI in late 2024.
    by woadwarrior01
  • This seems to be the demo:

    https://chat.webllm.ai/

    I am getting:

        WebGPUNotAvailableError: WebGPU is not supported in
    your current environment, but it is necessary to
    run the WebLLM engine.
    On both, FireFox and Chromium on Linux.
    by TekMol
  • You can enable WebGPU support in Google Chrome by turning on hardware acceleration and activating the WebGPU flag. It Works.
    by toto007
  • For firefox you may need to enable `dom.webgpu.enabled` in `about:config`

    It's on by default on Windows iirc - it's considered a potential security risk on Linux by Mozilla, so they ship it but it's turned off and it's up to the user to decide.

    I think the reasoning is just because of how varied graphics drivers/stacks are on Linux compared to Windows/OSX and the attack surface been larger.

    by noir_lord