Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- 35B MoE => 3B active parameters which basically means, the memory goes towards storing the various "expert" layers but the processing speed is only experiencing 3B.by cyanydeez