Everyone Should Know SIMD(mitchellh.com)
60 points by WadeGrimridge 2 hours ago | 4 comments
hnal943 9 minutes ago
[delayed]
eska 15 minutes ago
I don’t know zig syntax, but wouldn’t it be possible to put this common pattern into a macro and simplify it to mostly a lambda on V?
dnautics 5 minutes ago
no macros in zig, but yes you could metaprogram it. types are first class values at compile time so you could do that sort of specialization if you wanted.
wrl 16 minutes ago
i was having a conversation with a friend recently about simd in zig (which i have recently picked up and been having a pretty good time with). i find that simd writes decently well, though there's a few weird things:

- some builtins purport to work on simd vectors but actually just unpack the vectors and do their work per-element (e.g. running `@sin()` on a `@Vector(4, f32)` will unpack the vector, run `@sin()` 4 times, and then pack it back into a vector).

- a lot of `std.math` is scalar-only (some functions support vectors, though, and i've got a pr open for one of them and plan to do more).

- i'm certainly missing some intrinsics that i get from xmmintrin.h (rcp, rsqrt, few others).

in general though i'm finding it pretty capable.

mitchell, i know you hang around some of these comments sometimes – i noticed that in ghostty you bring in some c++ libs to do the simd heavy lifting for you. any plans to port that to zig? anything missing from the language or libs that's preventing it?

dnautics 2 minutes ago
> some builtins purport to work on simd vectors but actually just unpack the vectors and do their work per-element (e.g. running `@sin()` on a `@Vector(4, f32)` will unpack the vector, run `@sin()` 4 times, and then pack it back into a vector).

this is reasonable because there isn't really a generalizable "good way" to unroll trig functions for simd. if you really care about speed youre better off implementing to the precision you care about (you might not want full precision)

mitchellh 13 minutes ago
> mitchell, i know you hang around some of these comments sometimes

hi im here

> i noticed that in ghostty you bring in some c++ libs to do the simd heavy lifting for you. any plans to port that to zig? anything missing from the language or libs that's preventing it?

No plans to port it. For others, this is referencing highway: https://github.com/google/highway

The major limitation of Zig's vectors is that they're compile-time only. So if you're building redistributed software that compiles for a baseline CPU target, it won't be as optimized as it could be for YOUR possible machine.

Highway compiles our SIMD modules for different hardware configurations and at startup does a CPUID fingerprint to figure out which to load. That way even baseline has AVX512 etc. implementations, and we just activate the right one at runtime.

We only use Highway for our hottest hot paths that we feel benefit from that specialization.

No plans to port that (although, I spent hundreds of dollars and slop-forked it into Zig with the help of this good boy GPT and it worked great actually, but I didn't want to maintain it).

wrl 9 minutes ago
ahaaa, yeah, i don't personally do any runtime switching but i hear that as a deal-breaker from other folks.

it's interesting – i've found that zig tends to extend my vectors to the native width of the platform and then operate on them there. e.g. i had a `@Vector(2, f32)` that i was using as a demo and the generated assembly was promoting it to 256 bits and using avx2 instructions on it!

mitchellh 5 minutes ago
> i've found that zig tends to extend my vectors to the native width of the platform and then operate on them there

oh interesting. though i suspect that isn't zig and thats llvm.

qurren 47 minutes ago
I just do gcc -O3 and get SIMD without having to learn it
ashton314 35 minutes ago
In the article, Mitchel mentions how this doesn’t always work. In fact, as someone who’s worked in compiler development, I can say it’s a small miracle when it does work.
mitchellh 21 minutes ago
Case-in-point, the example in my own post doesn't auto-vectorize with LLVM or GCC at highest optimization levels. Basically, compilers will never auto-vectorize loops with an early loop break afaik.
forrestthewoods 41 minutes ago
auto-vectorization is not nearly as good as you would hope it to be.

The best SIMD optimizations likely require changing your data format from AoS to SoA.

ethin 17 minutes ago
Either this or you have to do special tricks like pairwise tree reductions and hand-unroll certain portions of loops.
nylonstrung 29 minutes ago
The one feature in Jonathan Blow's Jai language I really envy is a a single keyword to switch AoS to SoA and visa-versa at comptime
mbStavola 19 minutes ago
Didn't he drop this feature years ago?
raegis 34 minutes ago
What are AoS and SoA?
Georgelemental 30 minutes ago
Array of Structs and Struct of Arrays https://en.wikipedia.org/wiki/AoS_and_SoA
nylonstrung 30 minutes ago
Array of Structs and Struct of Arrays
formerly_proven 32 minutes ago
And -march=native or at least -march=x86-64-v3 or similar, alternatively identifying relevant functions and manually invoking FMV and uarch specialization via target_clones. Plus non-integer code can generally not be autovectorized in normal-math mode since FP is non-commutative.
Joker_vD 28 minutes ago
Well, then I just prompt Claude and get SIMD without having to learn it /s