Who is using FPGA for ML inference?

Join the discussion

• Hacker News
• No one because FPGAs are 10-20x less dense than ASICs actually designed for purpose. The tooling is also complete shit. Signed someone that did (part of) their PhD on this topic.
  by mathisfun123
• Train circuits, not weights; then use an FPGA
  by pugfugly
• You could build small ML model like talos-v2. But larger model/LLM requires much more engineering cost than you expected. Optimizing HDL has too many control knob/param to solve by RL.
  by b89kim
• For small models it is very do-able on entry level FPGA. With a PYNQ-Z2 at int8 you could potentially fit 1m params. It would be a fun exercise in writing a inference driver and implementing systolic arrays on a FPGA.
  by sphyrna-029
• ML inference using FPGA's is an active area of research, https://arxiv.org/html/2512.01463v1 and have been successfully used in LHC/HEP detectors such as CMS and ATLAS.
  by bschup
• I thought the same when I saw that the financial industry was hiring FPGA people for low-latency algorithms.

  My understanding (as a non-FPGA expert) is that currently FPGA beats generic hardware (CPU,GPU) for "small size algorithm" (i.e that do not need GB of weights), while enabling a certain flexibility vs ASIC.

  My guess is that you cannot bake all the weights into the circuit topology, so you are still bound by the memory transfer speed (to be double checked).

  by addag
• FPGAs are far more expensive than GPU+DRAM, even at today's inflated prices.
  by wmf