Birbla

Join the discussion

Write your take first — we'll ask for email only when you're ready to publish.

Sign in
  • Hacker News

  •   small native inference engine optimized first for DeepSeek V4 Flash (including the experimental vision model), DeepSeek V4.1 Flash (Metal, and text inference on CUDA), and additionally GLM 5.2 and 5.3, GLM 5.3 Flash and DeepSeek V4 PRO, and Qwen3.8 Flash Next (Metal and CUDA)
    This is local targeting high end consumer hardware like DGX Spark or AMD Ryzen AI Halo.

    For our mere mortals that were kids not long ago and can't really believe we've got our hands on a x090 series targeting Qwen3.8 27b, https://github.com/noonghunna/club-3090 is the way to go.

    I'm maintaining a web frontend for this, trying to at least. You can follow it here: https://github.com/gchamon/club-3090-server

    by gchamonlive
  • I’m a little confused

    ds4 is referring to “dwarfstar” “4” and references DeepSeek V4 most of the time

    but its model agnostic-ish

    and benchmarks compared to what? what do these large MoE models typically get in tokens per second?

    I’m garnering this is just an easier way to load large models per expert on consumer hardware? as opposed to the hackier solutions?

    I’m intruiged. Note that the blogpost says 64gb Macs are good minimums while the github says 96gb is a minimum

    by yieldcrv
  • The creator of this has a very interesting YouTube channel he posts to almost daily, talking mostly about current developments on AI from a technical but also societal/philosophical point of view. I specifically like that he provides a (much needed in this space) leftist point of view while not being anti-AI. Most of the videos are in Italian, so if you speak Italian (or are fine with YouTube automatic translation) I highly recommend it:

    https://youtube.com/@antirez

    by dudefeliciano
  • If you are interested in high-end models with high-end Apple Silicon, also try out Local Code: https://releases.drawthings.ai/p/public-beta-of-local-code-b... It is currently in TestFlight (and will open-source next week), supporting vision with DeepSeek 4.1 Flash, Qwen 3.8 27B and DeepSeek 4 Flash 0731 without vision. Custom quants & SSD streaming to make these big models work with 64GiB and above devices (and of course, Qwen works with devices with 16GiB and above).
    by liuliu
  • Ive been using this since it was initially released with deepseek v4 flash, and it is absolutely the best launcher ever on my m5 max 128gb

    Now Ive been running qwen 3.8 flash next for more than a week and it’s doing great, really fast and super long context windows. Sometimes the model is behaving stupidly by not remembering something I said earlier but it could be also a problem from the agentic AI harness. Im using oh my pi but Im wondering what people are using ds4 with here ?

    by ttoinou
  • Nothing comparable but inspired from DwarfStar I wrote a little inference engine for Intel Xe-LP (no XMX) 32GB laptops. The only model supported right now is a quantized Gemma-4, but I don't exclude in the future to support other MoE of similar size. Too bad we have no Qwen 3.8 35B-A3B yet.

    I'm also looking into expanding the protocol and the engine to support various steering techniques.

    https://github.com/simoneiacomino/xenolith

    by simoiacos
  • https://github.com/antirez/ds4

    The project GitHub page is a much better introduction for the hn crowd.

    by twoodfin
  • I maintain a fork of ds4 as shared libraries and thus can be used with other languages via FFI, along with public builds/binaries [1]. I made ds4go [2] against ds4 using techniques inspired by yzma.

    In addition to the library bindings, we have a small library of tools (workspace for view/edit, scratchpad for persistence) and making your own is registering a Go function. And in recent weeks, I added the Vision and Qwen support, as ds4 added them.

    Even if you don't use the Go library, the ds4go binary makes it really easy to download the libraries off of HuggingFace with a TUI available vie Homebrew.

    Here's some TUI toy screenshots, sorry I still haven't released that code; it's of different quality than the others. [3]

    EDIT: add ds4go TUI screenshot gist [4]

    [1] https://github.com/NimbleMarkets/ds4/releases/tag/v0.8.20260...

    [2] https://github.com/nimblemarkets/ds4go#install

    [3] https://gist.github.com/neomantra/ae47422c8daf7a458212c93992...

    [4] https://gist.github.com/neomantra/40180ade13df93290250ce8c6d...

    by neomantra

Explore Birbla archives