Will it be possible to do LLM Pool Training?

Will it be possible to do LLM Pool Training?

2 pointsby slicktux2 comments

Join the discussion

Write your take first — we'll ask for email only when you're ready to publish.

  • Hacker News
  • possible, yes, people already have projects

    practical, not yet, if ever (?)

    the difference is data and calculation dependencies, in crypto they are independent

  • Datacenter-scale GPGPU compute is extremely efficient compared to what you would achieve with a network of independently operated distributed heterogeneous devices spread across the world. Just imagine the overhead for ensuring that a node doesn't poison the gradients. And that's just one very small part of the problem.