Everyone should know SIMD

Join the discussion

• Hacker News
• I started learning multi-platform (x86 + ARM) SIMD last year by writing an audio synthesizer:

  https://github.com/seclorum/SIMDSynth

  It has been a very rewarding experience, and the synth architecture - multitimbral polyphonic - provides a great stream of data for applying SIMD principles, i.e. multiple streams going through the same process.

  Has been pretty hard to debug, though. I found myself wishing I had some sort of simulator to help me understand the state of things in each pipe. I suppose I should spend some time investigating SIMD tools next time I get into this - but I fear it'll require a lot more investment. If anyone has any tips, I'm all ears ..

  by MomsAVoxell
• I think an even better advice is that everyone should know array programming, because you generally need that mindset for SIMD optimizations as (packed) SIMD-specific techniques are surprisingly rare. And array programming gives you a generally performant code even without SIMD because it is much easier to auto-vectorize.
  by lifthrasiir
• I like SIMD, but before super-optimizing your code with SIMD and the like, really consider your data structures and access patterns.

  I've been singing Data-Oriented Design's praises, so I'll just collect all my comments here [1], but I think it's a good approach to optimization. I played around with SIMD in my old code (in Zig), but my approach to modelling datastructures was so antithetical to optimization, it was like putting high-performance racing tires on a lemon with a broken engine.

  It was the root-of-all-evil-type-premature-optimization, because I wasn't measuring performance, and I wasn't thinking about where the allocations were, etc. Now, I try to model my data as if it were SQL tables, see what my potential "primary keys" could be, and build my data structures around my access patterns.

  For example, I used to model trees as structs pointing to other structs on the heap:

      struct Tree {
          tag: TreeTag,
          children: Vec<&Tree>
      }
  

  Now my tree has all the bad characteristics of a linked list (* n nodes * m children), all the fragmentation of multiple heap vectors (* n nodes), and terrible set-up / tear-down time (in this case, Drop alone was taking up a good chunk of runtime).

  But a tree can be represented a million ways, and can always be linearized. So now I really consider my access/insert patterns of the tree, whether it's really a tree or some other sort of graph, whether I can store it in a Vec or a Struct of Vecs, etc. Since really looking at things through their access patterns and "primary keys", my code has been much faster and simpler.

  This has the added effect that a lot of your data ends up in homogeneous arrays / vecs, which means that the compiler can do its SIMD magic, the CPU can read it from your L1 cache a million times faster, etc. And then when you need to drop down into SIMD yourself, you can write some awesome branchless code.

  1. https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...

  by Rendello
• Here's a helpful video about leveraging SIMD to solve a concrete performance problem for the dev team that made the game The Witness by Casey Muratori: https://www.youtube.com/watch?v=Ge3aKEmZcqY
  by hnal943
• To bolster the argument, even if you do not plan to write the SIMD yourself or will "just get AI to do it", it is important to know what can be fast in SIMD (and on what hardware). That allows you to design your algorithms and structure your code so that the SIMD is possible.

  Internalizing things like how data dependencies matter, how expensive it is to increase the width of your vector elements (and how to avoid the need), how to turn conditions and branches into masks, or simply things like "division does not exist" becomes a lot easier when you have spent at least some time trying to use SIMD yourself.

  by derf_
• The last few days I've been using AVX-512 to optimize matrix operations in a bioinformatics project, and it's great! The bottleneck in most applications is reading the large dataset from memory, so rather than doing it multiple times to compute multiple operations you can do everything in one pass (fused kernel) with AVX registers. 5x speedups are quite common. I've been doing it with manual intrinsics, but the wide crate also makes common operations completely trivial. Highly recommend checking it out.

  https://docs.rs/wide/latest/wide/

  by jwgarber
• I'd slightly rephrase the title to "everyone should know when SIMD didn't happen." Modern compliers are extremely good at vectorization until they suddenly aren't, an they'll often fall back to scalar code because if assumptions or a single-data dependent branch. Learning to check the compliers optimization reports is arguably more valuable.
  by kiaansaraiya
• Good article!

  I just wouldn't start off with bold sentences as

  > SIMD can be simple to understand

  and

  > writing SIMD is just about as easy as a for loop

  and then the first example requires 12 lines to replace one line of scalar code.

  Be honest and say SIMD is hard but the results are worth it!

  (Another nitpick: if this article is for newbies, don't use SIMD-only words and concpts before explaining them. Step 5 is good: scalar tails are mentioned and described. Step 1 is bad: nobody is supposed to know what broadcast mean.)

  by teo_zero