Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- i was having a conversation with a friend recently about simd in zig (which i have recently picked up and been having a pretty good time with). i find that simd writes decently well, though there's a few weird things:
- some builtins purport to work on simd vectors but actually just unpack the vectors and do their work per-element (e.g. running `@sin()` on a `@Vector(4, f32)` will unpack the vector, run `@sin()` 4 times, and then pack it back into a vector).
- a lot of `std.math` is scalar-only (some functions support vectors, though, and i've got a pr open for one of them and plan to do more).
- i'm certainly missing some intrinsics that i get from xmmintrin.h (rcp, rsqrt, few others).
in general though i'm finding it pretty capable.
mitchell, i know you hang around some of these comments sometimes – i noticed that in ghostty you bring in some c++ libs to do the simd heavy lifting for you. any plans to port that to zig? anything missing from the language or libs that's preventing it?
by wrl - > some builtins purport to work on simd vectors but actually just unpack the vectors and do their work per-element (e.g. running `@sin()` on a `@Vector(4, f32)` will unpack the vector, run `@sin()` 4 times, and then pack it back into a vector).
this is reasonable because there isn't really a generalizable "good way" to unroll trig functions for simd. if you really care about speed youre better off implementing to the precision you care about (you might not want full precision)
by dnautics - > mitchell, i know you hang around some of these comments sometimes
hi im here
> i noticed that in ghostty you bring in some c++ libs to do the simd heavy lifting for you. any plans to port that to zig? anything missing from the language or libs that's preventing it?
No plans to port it. For others, this is referencing highway: https://github.com/google/highway
The major limitation of Zig's vectors is that they're compile-time only. So if you're building redistributed software that compiles for a baseline CPU target, it won't be as optimized as it could be for YOUR possible machine.
Highway compiles our SIMD modules for different hardware configurations and at startup does a CPUID fingerprint to figure out which to load. That way even baseline has AVX512 etc. implementations, and we just activate the right one at runtime.
We only use Highway for our hottest hot paths that we feel benefit from that specialization.
No plans to port that (although, I spent hundreds of dollars and slop-forked it into Zig with the help of this good boy GPT and it worked great actually, but I didn't want to maintain it).
by mitchellh - Tangentially for Go programming, the last time I looked at optimising some Go code with SIMD there were a few different options available, but they were either not maintained any more or had incomplete support and required first writing your function in C++ with intrinsics and generating assembly, then converting it to go assembly with a tool [1]. I never got my function to work in go despite the C++ code working fine. In short, not really a production ready option for Go. This was a year or two ago, though.
Edit, there's now an experimental official library at https://go.dev/pkg/simd/archsimd/ see https://go.dev/doc/go1.26#simd and at https://github.com/golang/go/issues/78902 so things have moved since I tried it last.
by kristianp - It actually works really well in the last couple Go versions with GOEXPERIMENT=simd. You do get a similar speedup (if not higher, since SIMD also eliminates the penalty for bounds checking and other things Go runtime does.by nasretdinov
- > Every developer should… most importantly, not be scared of SIMD
Seems like he should be recommending fearless_simd [1], the Rust crate by Raph Levian and the folks at Linebender :)
More seriously, if you’re looking to add SIMD to your Rust code, that’s the package to start with.
by arijun - > Every developer should know at least that much SIMD.
> This [...] applies to any programming language. Support for SIMD instructions varies by programming language
This is a very pedantic nitpick because this article is good (& getting SIMD support across more languages would also be good), but the "every programmer should know" line feels a bit odd when neither of the 2 most popular languages natively support SIMD.
by lucideer - I would venture as far as to say that the most popular languages might not be the most used by software engineers, as in, people this kind of article would be aimed at.by lionkor
- I think an even better advice is that everyone should know array programming, because you generally need that mindset for SIMD optimizations as (packed) SIMD-specific techniques are surprisingly rare. And array programming gives you a generally performant code even without SIMD because it is much easier to auto-vectorize.by lifthrasiir
- Array programming where we compare first and look for the first failure later will not help much here if runs are short because by itself it doesn't give you early termination and you may spend a lot of time on wasted comparisons.by macleginn
- I'm no fan of closed-source languages, and lord knows MATLAB has its warts. But I can't deny that it was pretty seamless to write efficient vectorised code for numerical simulations at uni. I don't have much experience with it, but my understanding is that Julia is the closest thing to a more modern and expressive language that has similar vectorisation capabilities.by subset
- I like SIMD, but before super-optimizing your code with SIMD and the like, really consider your data structures and access patterns.
I've been singing Data-Oriented Design's praises, so I'll just collect all my comments here [1], but I think it's a good approach to optimization. I played around with SIMD in my old code (in Zig), but my approach to modelling datastructures was so antithetical to optimization, it was like putting high-performance racing tires on a lemon with a broken engine.
It was the root-of-all-evil-type-premature-optimization, because I wasn't measuring performance, and I wasn't thinking about where the allocations were, etc. Now, I try to model my data as if it were SQL tables, see what my potential "primary keys" could be, and build my data structures around my access patterns.
For example, I used to model trees as structs pointing to other structs on the heap:
Now my tree has all the bad characteristics of a linked list (* n nodes * m children), all the fragmentation of multiple heap vectors (* n nodes), and terrible set-up / tear-down time (in this case, Drop alone was taking up a good chunk of runtime).struct Tree { tag: TreeTag, children: Vec<&Tree> }But a tree can be represented a million ways, and can always be linearized. So now I really consider my access/insert patterns of the tree, whether it's really a tree or some other sort of graph, whether I can store it in a Vec or a Struct of Vecs, etc. Since really looking at things through their access patterns and "primary keys", my code has been much faster and simpler.
This has the added effect that a lot of your data ends up in homogeneous arrays / vecs, which means that the compiler can do its SIMD magic, the CPU can read it from your L1 cache a million times faster, etc. And then when you need to drop down into SIMD yourself, you can write some awesome branchless code.
1. https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...
by Rendello - This. Tables are an efficient implementation of general graphs. It’s the best one I know of (unless your graph can be specialized).
- Very much agreed. Even more basic than that - memory access patterns are important. The amusing thing is that you end up writing GPU-style code even for CPU. For example - instead of an array of objects, using parquet-style object of arrays is one such trick.by Scene_Cast2
- This is my eternal battle as a perf engineer. Performance starts with architecture and you can only squeeze so much out a hotpath with poor data layout.
The nice part is that data-oriented code almost always easily supports threading and SIMD.
by luaKmua - Yeah, data layout/cache aware layouts are really key if you really want to unlock making something that ends up in a hot loop fast with SIMD.
Also, avoiding allocations or vtable lookups or a lot of indirection in the part of the code that's actually "hot" is really important. Vectors (in C++) at least aren't necessarily the best fit either, if you end up doing anything that can call an allocation unexpectedly.
by tarnith - I started learning multi-platform (x86 + ARM) SIMD last year by writing an audio synthesizer:
https://github.com/seclorum/SIMDSynth
It has been a very rewarding experience, and the synth architecture - multitimbral polyphonic - provides a great stream of data for applying SIMD principles, i.e. multiple streams going through the same process.
Has been pretty hard to debug, though. I found myself wishing I had some sort of simulator to help me understand the state of things in each pipe. I suppose I should spend some time investigating SIMD tools next time I get into this - but I fear it'll require a lot more investment. If anyone has any tips, I'm all ears ..
by MomsAVoxell - Here's a helpful video about leveraging SIMD to solve a concrete performance problem for the dev team that made the game The Witness by Casey Muratori: https://www.youtube.com/watch?v=Ge3aKEmZcqYby hnal943
- It's a great talk, I just wish there was a good focused textual version of it, as it is a very long video to recommend to others. Very worth it, but a big investment.
It's a great example of what I think of as vertical integration for performance. As you go through the talk you can understand why all these abstractions exist and why they have to be so generic. But when you have a specific use case, you can vertically integrate from the problem definition all the way down to SIMD and reap big rewards.
by crabmusket - To bolster the argument, even if you do not plan to write the SIMD yourself or will "just get AI to do it", it is important to know what can be fast in SIMD (and on what hardware). That allows you to design your algorithms and structure your code so that the SIMD is possible.
Internalizing things like how data dependencies matter, how expensive it is to increase the width of your vector elements (and how to avoid the need), how to turn conditions and branches into masks, or simply things like "division does not exist" becomes a lot easier when you have spent at least some time trying to use SIMD yourself.
by derf_ - > what can be fast
I think this doesn't get talked about enough. If your input is a big run of data that is being checked/transformed in one shot, it works well. But, if you're likely to have to make a decision on several bytes of the input, SIMD will be the same or slower than the scalar method. It's not a magic "go fast" button.
by tstack - The last few days I've been using AVX-512 to optimize matrix operations in a bioinformatics project, and it's great! The bottleneck in most applications is reading the large dataset from memory, so rather than doing it multiple times to compute multiple operations you can do everything in one pass (fused kernel) with AVX registers. 5x speedups are quite common. I've been doing it with manual intrinsics, but the wide crate also makes common operations completely trivial. Highly recommend checking it out.by jwgarber
- I'd slightly rephrase the title to "everyone should know when SIMD didn't happen." Modern compliers are extremely good at vectorization until they suddenly aren't, an they'll often fall back to scalar code because if assumptions or a single-data dependent branch. Learning to check the compliers optimization reports is arguably more valuable.by kiaansaraiya
- > Learning to check the compliers optimization reports is arguably more valuable
Where do I start?
I want to trust the compiler, but I don't always have time to feed every little piece into compiler explorer and interpret it. Is there a higher-level workflow?
by randusername - > Learning to check the compliers optimization reports is arguably more valuable.
More valuable than learning to write SIMD code? When writing SIMD code is itself the remedy to lousy autovectorisation?
If you can only identify the problem, you’re left with “well, that sucks”.
by spider-mario - That’s exactly what happened:by alberth
- Good article!
I just wouldn't start off with bold sentences as
> SIMD can be simple to understand
and
> writing SIMD is just about as easy as a for loop
and then the first example requires 12 lines to replace one line of scalar code.
Be honest and say SIMD is hard but the results are worth it!
(Another nitpick: if this article is for newbies, don't use SIMD-only words and concpts before explaining them. Step 5 is good: scalar tails are mentioned and described. Step 1 is bad: nobody is supposed to know what broadcast mean.)
by teo_zero - I had the same thoughts about SIMD code being too verbose when I wrote some, so a few months ago I tried writing a library that lets you write quasi-GLSL code in C++, so much more compact, with the ability to switch between SIMD width without having to rewrite anything at all:
https://github.com/gitdepierre/cppshader
Not sure it will ever be useful, but it was a fun pet project with some interesting problems to solve.
by relssiegp - It's all relative though. The rules of playing bridge (the card game) are much harder than understanding the rules around SIMD instructions.by amelius
- Agreed, I was interested and I'm prob the target audience but things escalated too fast too quickly, very similar to the infamous "how to draw an owl" meme.by franciscop