Birbla

Join the discussion

Write your take first — we'll ask for email only when you're ready to publish.

Sign in
  • Hacker News
  • how is this different from ollama? for me the more/open the merrier.
    by synergy20
  • While this is very cool and llamafiles are quite universal there is anyway a nuance for Window systems which is the size limit for a Windows executable which is 4Gb maximum. As LLM models are tend to be quite large this limit is reached pretty fast. So for such cases llamafile.exe will be required (which is also universal and runs everywhere). And at the end it could be just llama.cpp tools which released for all platforms + the LLM model file itself.
    by romperstomper
  • Cosmocc and Cosmopolitan are remarkable technical achievements and llamafile made me discover them.

    The llamafile UX (CLI interface and web server with chat to quickly interact with the model) is great and make easy to download and play with a local LLM.

    However I fail to see use cases where I would build a solution built on a llamafile. If I want to play with multiple models, I don't need to have the binary attached to the model data. If I want to play with a model on multiple operating systems, I'm fine downloading the llamafile tool binary for the platform separately from the model data (in fact, on Windows one have to download the llamafile.exe separately anyway because of a limit of the OS for executable files).

    So Cosmopolitan is great tech, the llamafile command (the "UX for a model" part) is great, but I'm not convinced by the value of Cosmopolitan applied here.

    by dolmen
  • I’m glad to see llamafile being resurrected. A few things I hope for:

    1. Curate a continuously extended inventory of prebuilt llamafiles for models as they are released 2. Create both flexible builds (with dynamic backend loading for cpu and cuda) and slim minimalist builds 3. Upstreaming as much as they can into llama.cpp and partner with the project

    by michaelgiba
  • Tips:

        # Avoid issues when wine is installed.
    sudo su -c 'echo 0 > /proc/sys/fs/binfmt_misc/status'
    And:

        # Capture the entirety of the instructions to obtain the input length.
    readonly INSTRUCT=$(
      join ${PATH_PREFIX_SYSTEM} ${PATH_PROMPT_SYSTEM} ${PATH_PREFIX_SYSTEM}
      join ${PATH_SUFFIX_USER} ${PATH_PROMPT_USER} ${PATH_SUFFIX_USER}
      join ${PATH_SUFFIX_ASSIST} "/dev/null" ${PATH_SUFFIX_ASSIST}
    )

    (
      echo ${INSTRUCT}
    ) | ./llamafile \
      -m "${LINK_MODEL}" \
      -e \
      -f /dev/stdin \
      -n 1000 \
      -c ${#INSTRUCT} \
      --repeat-penalty 1.0 \
      --temp 1.5 \
      --silent-prompt > output.txt
    by thangalin
  • justine tunney gave a great intro to Llamafile at AIE last year if it helps anyone: https://www.youtube.com/watch?v=-mRi-B3t6fA
    by swyx
  • > As the local and open LLM ecosystem has evolved over the years, time has come for llamafile to evolve too. It needs refactoring and upgrades to incorporate newer features available in llama.cpp and develop a refined understanding of the most valuable features for its users.

    It seems people have moved on from Llamafile. I doubt Mozilla AI is going to bring it back.

    This announcement didn't even come with a new code commit, just a wish. https://github.com/mozilla-ai/llamafile/commits/main/

    by benatkin
  • Really exciting to see Mozilla AI starting up and I can't wait to see where the next generation takes the project!
    by jart

Explore Birbla archives