Birbla

MacBook vs. Dedicated GPU for LLM

MacBook vs. Dedicated GPU for LLM

22 pointsby mzubairtahir71 comments

Join the discussion

Write your take first — we'll ask for email only when you're ready to publish.

Sign in
  • Hacker News
  • Both are going to be super super slow and low payback.

    You gotta really want it right now.

    It's still early!

    by epsteingpt
  • It depends. I have a M5 128 so i can play around with large models and even keep several of them loaded at once and use something like llama swap to access them all via bifrost or litellm. You won't do this without some serious local GPU's with big memory. The downside is the speed, it's not fast, but fast enough to tinker and develop with worrying about ongoing cloud cost. When done developing and you need to really scale up this is when you can swap to cloud computing and get the job done faster. My $5k macbook can do more than a $50k nvidia/intel/amd setup, just not as fast.

    So you need to decide whats important to you if you want to work locally, large/many models or speed. It's the pick 2 problem speed, size, cost, pick 2 or go with cloud and accept your development time is also spent $$ on each iteration.

    I was hoping for the M5 ultra by now, but looks like that's not coming until much later this year for a much higher price now.

    by hkchad
  • Around February or march I started looking into hardware options to help me start learning about training models and working with them. My budget was limited and an apple refurbished 32 gb Mac mini was far and away the best option for my budget. I wish it was faster but I can let it run 24/7 with no noise and minimal power draw. I just arrange long running tasks for when am asleep or at work. Then as a huge plus I have an awesome daily driver machine for whatever else I want to do
    by g-technology
  • With a M5 16c 48GB and Qwen 3.6 35B Q4 I get up to 1900 PP/s and 80 TG/s. With an Nvidia 5090 I get 7800 PP/s and 280 TG/s.

    Together with pi mono I wouldn't want to go back to Claude & Co. Speed, quality of the answers, short answer times at any time of day - once you have eaten from the fruit your definition of SOTA will change...

    For reference, I do software development since 30 years, I am not vibe coding the umpteenth todo list.

    by dust42
  • My opinion is that you should wait for 6-12 months before making a purchase either way.

    Open weight models are getting good. With GLM 5.2 now chasing Opus, I'm very excited to see a smaller model's distillation.

    Plus, the OLED MacBook Pro should be released by then.

    by nichch
  • MacBooks with their unified memory behave like a slow GPU with enormous amount of video RAM. So you can run large smart models slowly.

    Dedicated GPUs have less video RAM so can run smaller less smart models quickly.

    by JSR_FDED
  • If you want a massive MacBook anyway then it's great. They are decent for local LLMs, awesome for local image models and it's a MacBook so AppleCare+ has your back. IMO it's a no brainer if you wanted a MacBook anyway but it's a poor choice if your reason to buy it is to run LLMs.
    by jpgvm
  • I asked a few of my friends that are ML engineers this question and all of them said to run the LLMs in the cloud with their infrastructure because it was going to be way faster. If you just want to tinker around I would look at @JSR_FDD's comment.
    by cylentwolf

Explore Birbla archives