Will it be possible to do LLM Pool Training?

Will it be possible to do LLM Pool Training?

2 pointsby slicktux2 comments

Comments

Hacker News

possible, yes, people already have projects

practical, not yet, if ever (?)

the difference is data and calculation dependencies, in crypto they are independent

by verdverm

Datacenter-scale GPGPU compute is extremely efficient compared to what you would achieve with a network of independently operated distributed heterogeneous devices spread across the world. Just imagine the overhead for ensuring that a node doesn't poison the gradients. And that's just one very small part of the problem.

by bjourne

Join the discussion

Write your take first — we'll ask for email only when you're ready to publish.

  • Hacker News
  • possible, yes, people already have projects

    practical, not yet, if ever (?)

    the difference is data and calculation dependencies, in crypto they are independent

  • Datacenter-scale GPGPU compute is extremely efficient compared to what you would achieve with a network of independently operated distributed heterogeneous devices spread across the world. Just imagine the overhead for ensuring that a node doesn't poison the gradients. And that's just one very small part of the problem.
Will it be possible to do LLM Pool Training? · Birbla