This is quite amazing. Fearless SIMD is what finally allowed my FFT crate, PhastFT, to run on stable Rust [0]. PhastFT previously depended on Rust’s std::simd (as well as multiversion), which still requires nightly [1].
One of the main contributors to v1.0 of fearless_simd started out by helping me evaluate portable SIMD crates for PhastFT. Once we landed on fearless_simd, he ported PhastFT from std::simd to fearless_simd. We did find that more functionality was needed than fearless_simd offered at the time. So, he started contributing significantly to fearless_simd. It’s really rewarding to see how working on a hobby open source project can help improve the Rust ecosystem.
Does Rust or any other language support customizing the compiler so that interprocedural analyses can track custom subsets of, for example, doubles so that the compiler can choose the most efficient instruction sequence for example for min/max? If we know a double is never NaN then we can emit only one instruction on x86, but have to emit one more on arm64. If we know a double is never zero and never NaN, we can emit a single instruction on both.
This whole conversation between relaxed SIMD and deterministic SIMD seems to only exist because our compilers are not smart enough and/or their whole program analyses don’t support any plugin-like capabilities.
There are other examples where if we know a SIMD bitmask is canonical (all 1s per lane) then we can implement horizontal reductions more efficiently. This is very niche and I doubt that any language supports interprocedural analyses with such a rich domain, so it feels like a hole in the programming language space.
Not yet that I know of, though perhaps LLVM might be able to infer simple cases (when loading from a known constant for example).
Pattern types could potentially maybe in the future allow for the compiler to know more details though. They are a nightly feature, and afaik only for enums, integers and pointers so far. The idea would be that you can define a custom type such as "an integer between 7 and 45" and everything else become niches for niche optimisation (e.g. for `Option<MyFunkyInt>` some of those impossible values would be used to represent the None case of the wrapping Option).
But I could envisage a future in which you could say "f64 without NaN" which would both make those available for niches and potentially tell LLVM about this. However, we are very far from any of that currently. And it might not be what you want, since you would need to add checks when you perform operations to ensure the value doesn't suddenly become a NaN. Which is way more complicated than ensuring integers don't become, say, zero. It will likely be much harder to optimise away the checks.
LLVM has range flags and things like nnan that could in theory be used to replace a minimum intrinsic into a minimumnum (iirc x86 has a instruction for the latter but not the former) https://llvm.org/docs/LangRef.html#floating-point-min-max-in... I'm not sure if the optimizations use this but in theory they could
That is a big maybe though. It is very experimental (didn't even have non-placeholder syntax last I looked), and as far as I know nobody has yet even discussed it for floats.
But I suspect you're overvaluing the potential savings. Knowing when a float is 0.0 or NaN beforehand is almost entirely impossible, except for the most trivial of cases - like when you first initialize a variable or first enter a loop. Everything after that is very hard or impossible with floats as they are.
Those cases can be const folded at compile time.
Those cases are never a measurable bottleneck.
The closest thing I know of in the realm of the optimization you're curious about is Rust NonZero* variants, but they're used for enum compression afaik.
I’m not sure I agree on the impossible part, I feel like a sufficiently smart interprocedural analysis that also implements range analysis interprocedurally could prove a lot to where it becomes useful.
I guess what I would like to see is SIMD libraries being able to confidently say nobody needs to use intrinsics (or differentiate between relaxed/normal SIMD on the user API level) because the language + high level SIMD APIs are smart enough to choose the right implementation.
IIRC IEEE min/max with proper NaN handling needs 8 instructions on x86 vs 1 on arm64 I find it very sad that we apparently haven’t really solved that yet without forcing the user to use different APIs.
That vminpd+vminpd+vorpd actually does handle signed zero properly! Screws up NaN payloads to the max though. Can be easily extended to canonicalize the NaN with 2 instrs + constant though of course. (which ends up at the same number of instrs as your proper impl (albeit with worse port distribution and latency), but you get to have a canonical NaN!)
Hit upon https://github.com/llvm/llvm-project/issues/217376 while playing around with proper minimumnum, failing to SMT-verify whatever version of LLVM I had; did find a funky working 6-instr (+ constant) version though:
vpandn ymm2, ymm1, ymm0
vpcmpeqd ymm2, ymm2, 0x80000000 # whether ymm0 is -0 and ymm1 is +0 (or other cases that magically don't cause issues)
vcmpltpd ymm3, ymm0, ymm1
vcmpunordpd ymm3, ymm3, ymm1 # regular NaN-is-larger ymm0<ymm1
vpor ymm2, ymm2, ymm3
vpblendvb ymm0, ymm1, ymm0, ymm2
NonZerof32::from_bits(1) multiplied with itself is zero.
Doing range analysis needs the language to support it at compile time, and the dev to specify what range it is.
The only 'stable' thing i can think of is a type for 'greater-eq-one' using only addition and multiplication. Practically every other operation breaks most of the type knowledge up to that point.
Funnily enough, intrinsics have become almost trivial to write, with agentic tooling. I used to be an ISPC advocate but now I'm finding myself gravitate to direct use of intrinsics with bespoke dynamic/runtime dispatching more than ever.
Yes, Highway is pretty nice but also quite elaborate when it comes to dealing with multiple vectorized versions and dispatch. The macros burn my eyes still.
Customized compiler isn't really a thing in most languages. Because it implies changing the language, and everyone agreeing on what the language is was sort of the point of the whole exercise. Hence all the worries about macros.
SBCL has the right hooks. Jai probably does as well. You could build your own thing on LLVM relatively easily - so Rust _could_ do it if you gave it sufficient access to the compiler, but so could C++ or D.
SIMD bitmasks being ~0 instead of (00000001)+ is common and useful, but there's an annoying language design question in there about what type comparisons between vectors should be (if you don't have a <N x i1> as a type, say because you liked C too much). Shout out to std::vector<bool>.
There is probably interesting work in mojo for this. The library being in MLIR strongly encourages taking that sort of approach. I haven't looked at their implementation though. Happy hacking!
Julia also does this quite often (both using macros to write DSLs or minor modifications, and packages like GPU backends that hook the compiler functionality to compile direct GPU assembly kernels.
Have been really happy with Fearless simd, I used it to write [memchr-n][1], which searches for instances of an arbitrary set of bytes, with simd, which actually tends to [outperform][2] the rust memchr crate, even for sets of 1-3 bytes (what's supported by memchr)
Kinda:
Early last year, I had done a little work with direct intrinsics first, then investigated std::simd, but ran into the issue of swizzle_dyn needing to re-compile std. It was actually when I ran across https://shnatsel.github.io/improving-std-simd-swizzle-dyn/ that got me back into it.
I think projects that seemingly don't even use Semver (putty uses two instead of three numbers), and also were created before semver was even a "concrete thing" (~2010?), gets a pass on the requirements that semver proposed, like the whole "1.0.0 means stable and in wide use" thing.
Semver or otherwise, I think even decades ago people understood staying on 0. for years or decades is a bit silly for production software tens of thousands of people use every day.
Doing pacman -Q | grep ' 0\.' locally I get 547 hits, with alacritty, bubblewrap, dconf, fzf, hyprland, mtr, neovim and more among them, software that is in wide use already, some of them quite old (dconf apparently almost two decades old).
I think people read too much into what is just a number for some projects, while for others (like semver zealots), the numbers carry actual meaning.
And it would probably be 0.4xx-something if they had their current release numbering policy from day 1 instead of all the piddling around with u releases in the 90s and 2000s and even up a little bit to the early 2010s.
The Rust project was initiated to solve two thorny problems:
- How do you do safe systems programming?
- How do you make concurrency painless?
Initially these problems seemed orthogonal, but to our amazement, the solution turned out to be identical: the same tools that make Rust safe also help you tackle concurrency head-on.
That was an eye-opener at the time, at least for me. Maybe Fearless SIMD borrows it's name from that sentiment.
I started an early Rust SIMD crate with similar aims (SIMDeez) and so I know how hard it is to do this well, so wanted to say congratulations to everyone who worked on Fearless SIMD. Tools like this are really nice for being able to leverage the gigantic performance modern CPUs make available without having to write intrinsics for each platform.
This is quite amazing. Fearless SIMD is what finally allowed my FFT crate, PhastFT, to run on stable Rust [0]. PhastFT previously depended on Rust’s std::simd (as well as multiversion), which still requires nightly [1].
One of the main contributors to v1.0 of fearless_simd started out by helping me evaluate portable SIMD crates for PhastFT. Once we landed on fearless_simd, he ported PhastFT from std::simd to fearless_simd. We did find that more functionality was needed than fearless_simd offered at the time. So, he started contributing significantly to fearless_simd. It’s really rewarding to see how working on a hobby open source project can help improve the Rust ecosystem.
[0] https://github.com/smu160/PhastFT
[1] https://doc.rust-lang.org/std/simd/index.html
How do you compare your experience with fearless_simd vs std::simd? Besides the nightly aspect of course.
I’m currently working on adding better SIMD support to Dart (https://github.com/dart-lang/sdk/issues/64170) and I have a question for the author or others here.
Does Rust or any other language support customizing the compiler so that interprocedural analyses can track custom subsets of, for example, doubles so that the compiler can choose the most efficient instruction sequence for example for min/max? If we know a double is never NaN then we can emit only one instruction on x86, but have to emit one more on arm64. If we know a double is never zero and never NaN, we can emit a single instruction on both.
This whole conversation between relaxed SIMD and deterministic SIMD seems to only exist because our compilers are not smart enough and/or their whole program analyses don’t support any plugin-like capabilities.
There are other examples where if we know a SIMD bitmask is canonical (all 1s per lane) then we can implement horizontal reductions more efficiently. This is very niche and I doubt that any language supports interprocedural analyses with such a rich domain, so it feels like a hole in the programming language space.
Not yet that I know of, though perhaps LLVM might be able to infer simple cases (when loading from a known constant for example).
Pattern types could potentially maybe in the future allow for the compiler to know more details though. They are a nightly feature, and afaik only for enums, integers and pointers so far. The idea would be that you can define a custom type such as "an integer between 7 and 45" and everything else become niches for niche optimisation (e.g. for `Option<MyFunkyInt>` some of those impossible values would be used to represent the None case of the wrapping Option).
But I could envisage a future in which you could say "f64 without NaN" which would both make those available for niches and potentially tell LLVM about this. However, we are very far from any of that currently. And it might not be what you want, since you would need to add checks when you perform operations to ensure the value doesn't suddenly become a NaN. Which is way more complicated than ensuring integers don't become, say, zero. It will likely be much harder to optimise away the checks.
LLVM has range flags and things like nnan that could in theory be used to replace a minimum intrinsic into a minimumnum (iirc x86 has a instruction for the latter but not the former) https://llvm.org/docs/LangRef.html#floating-point-min-max-in... I'm not sure if the optimizations use this but in theory they could
Sounds similar to https://mlir.llvm.org/
> Pattern types could potentially maybe in the future allow for the compiler to know more details though.
Thanks, I’ll take a look!
That is a big maybe though. It is very experimental (didn't even have non-placeholder syntax last I looked), and as far as I know nobody has yet even discussed it for floats.
Last time i checked; no.
But I suspect you're overvaluing the potential savings. Knowing when a float is 0.0 or NaN beforehand is almost entirely impossible, except for the most trivial of cases - like when you first initialize a variable or first enter a loop. Everything after that is very hard or impossible with floats as they are.
Those cases can be const folded at compile time.
Those cases are never a measurable bottleneck.
The closest thing I know of in the realm of the optimization you're curious about is Rust NonZero* variants, but they're used for enum compression afaik.
I’m not sure I agree on the impossible part, I feel like a sufficiently smart interprocedural analysis that also implements range analysis interprocedurally could prove a lot to where it becomes useful.
I guess what I would like to see is SIMD libraries being able to confidently say nobody needs to use intrinsics (or differentiate between relaxed/normal SIMD on the user API level) because the language + high level SIMD APIs are smart enough to choose the right implementation.
IIRC IEEE min/max with proper NaN handling needs 8 instructions on x86 vs 1 on arm64 I find it very sad that we apparently haven’t really solved that yet without forcing the user to use different APIs.
You can do it with just 3 instructions for IEEE 754-2019 minimumNumber (ignores NaN):
If you want proper IEEE 754-2019 minimum (propagate NaN, -0.0 < +0.0, NaN bitpattern picked in the usual way) you can do it in 6: I personally find this a load of nonsense I don't care about.If you want propagating NaNs but don't care about signed zero or NaN payload/sign, you can use
What I do in Polars is a bit different, there for propagating NaNs I do this isn't fully optimal on x86-64 but it's fairly simple and autovectorizes decently on various platforms, here's AVX2:That vminpd+vminpd+vorpd actually does handle signed zero properly! Screws up NaN payloads to the max though. Can be easily extended to canonicalize the NaN with 2 instrs + constant though of course. (which ends up at the same number of instrs as your proper impl (albeit with worse port distribution and latency), but you get to have a canonical NaN!)
Hit upon https://github.com/llvm/llvm-project/issues/217376 while playing around with proper minimumnum, failing to SMT-verify whatever version of LLVM I had; did find a funky working 6-instr (+ constant) version though:
Stop the presses! 5 instructions!
yay for superoptimizersAnything without range analysis is not worth it.
Note that:
NonZerof32 * NonZerof32 -> NonNanf32
NonZerof32::from_bits(1) multiplied with itself is zero.
Doing range analysis needs the language to support it at compile time, and the dev to specify what range it is.
The only 'stable' thing i can think of is a type for 'greater-eq-one' using only addition and multiplication. Practically every other operation breaks most of the type knowledge up to that point.
> nobody needs to use intrinsics
Funnily enough, intrinsics have become almost trivial to write, with agentic tooling. I used to be an ISPC advocate but now I'm finding myself gravitate to direct use of intrinsics with bespoke dynamic/runtime dispatching more than ever.
+1 on agents making library/DIY a lot more attractive than bespoke compilers.
In C++, one can also have portable intrinsics plus simpler runtime dispatching using our Highway library :)
> using our Highway library
Yes, Highway is pretty nice but also quite elaborate when it comes to dealing with multiple vectorized versions and dispatch. The macros burn my eyes still.
:) Yeah, those are best copy-pasted from existing code.
I do think we've converged on the best that's possible in C++.
Customized compiler isn't really a thing in most languages. Because it implies changing the language, and everyone agreeing on what the language is was sort of the point of the whole exercise. Hence all the worries about macros.
SBCL has the right hooks. Jai probably does as well. You could build your own thing on LLVM relatively easily - so Rust _could_ do it if you gave it sufficient access to the compiler, but so could C++ or D.
SIMD bitmasks being ~0 instead of (00000001)+ is common and useful, but there's an annoying language design question in there about what type comparisons between vectors should be (if you don't have a <N x i1> as a type, say because you liked C too much). Shout out to std::vector<bool>.
There is probably interesting work in mojo for this. The library being in MLIR strongly encourages taking that sort of approach. I haven't looked at their implementation though. Happy hacking!
Julia also does this quite often (both using macros to write DSLs or minor modifications, and packages like GPU backends that hook the compiler functionality to compile direct GPU assembly kernels.
LLVM does do bit pattern tracking. Referred to as value tracking or known bits analysis.
I’m not sure if the vectorizer uses it a ton, but isel does (though maybe not heavily for vector ops)
Possible in Julia without patching the compiler with a compiler extension package.
I think you are looking for NotNan<f64> from ordered-float crate?
Have been really happy with Fearless simd, I used it to write [memchr-n][1], which searches for instances of an arbitrary set of bytes, with simd, which actually tends to [outperform][2] the rust memchr crate, even for sets of 1-3 bytes (what's supported by memchr)
[1]: https://github.com/Dr-Emann/memchr_n#performance [2]: https://github.com/BurntSushi/memchr/pull/241
Out of curiosity, did you try out any other portable simd crates before landing on fearless simd?
Kinda: Early last year, I had done a little work with direct intrinsics first, then investigated std::simd, but ran into the issue of swizzle_dyn needing to re-compile std. It was actually when I ran across https://shnatsel.github.io/improving-std-simd-swizzle-dyn/ that got me back into it.
Related. Others?
Towards fearless SIMD, 7 years later - https://news.ycombinator.com/item?id=43519823 - March 2025 (175 comments)
Towards fearless SIMD - https://news.ycombinator.com/item?id=18293209 - Oct 2018 (81 comments)
Pretty happy this broke the 0.x curse!
The default system for Rust libs is ZeroVer: https://0ver.org/
PuTTY out since 1999 and bumped to 0.82 in 2024 is pretty incredible
I think projects that seemingly don't even use Semver (putty uses two instead of three numbers), and also were created before semver was even a "concrete thing" (~2010?), gets a pass on the requirements that semver proposed, like the whole "1.0.0 means stable and in wide use" thing.
Semver or otherwise, I think even decades ago people understood staying on 0. for years or decades is a bit silly for production software tens of thousands of people use every day.
Doing pacman -Q | grep ' 0\.' locally I get 547 hits, with alacritty, bubblewrap, dconf, fzf, hyprland, mtr, neovim and more among them, software that is in wide use already, some of them quite old (dconf apparently almost two decades old).
I think people read too much into what is just a number for some projects, while for others (like semver zealots), the numbers carry actual meaning.
For applications SemVer matters way less.
MAME was started in 1997 and is now on version 0.289 xD
And it would probably be 0.4xx-something if they had their current release numbering policy from day 1 instead of all the piddling around with u releases in the 90s and 2000s and even up a little bit to the early 2010s.
On the other hand, Retroarch users will keep running MAME 2003-Plus until the sun becomes a red giant...
It's exactly what we were doing before the semver hype cycle, but with 0. in front!
My outsider understanding is that lots of things make 1.0. Everytime something is abandoned.
Sounds more like a book title than the name of a library, to be honest.
Reminds me of the "Fearless Concurrency" blog post by Aaron Turon in 2015 (https://blog.rust-lang.org/2015/04/10/Fearless-Concurrency/).
Quote:
The Rust project was initiated to solve two thorny problems:
- How do you do safe systems programming?
- How do you make concurrency painless?
Initially these problems seemed orthogonal, but to our amazement, the solution turned out to be identical: the same tools that make Rust safe also help you tackle concurrency head-on.
That was an eye-opener at the time, at least for me. Maybe Fearless SIMD borrows it's name from that sentiment.
Edit: the original announcement says this is the inspiration (https://raphlinus.github.io/rust/simd/2018/10/19/fearless-si...)
rust has been around for a while and has collected a large number of crates (many of them abandoned). So new crates need to adopt new and clever names
This crate isn't even remotely new, it's from 2018.
just put a "new" or "better" in front of whichever name you want ;)
What is next, movie titles?
Wuthering SIMD
Fifty shades of SIMD
I started an early Rust SIMD crate with similar aims (SIMDeez) and so I know how hard it is to do this well, so wanted to say congratulations to everyone who worked on Fearless SIMD. Tools like this are really nice for being able to leverage the gigantic performance modern CPUs make available without having to write intrinsics for each platform.
How does the crate’s SIMD performance compare to ISPC, which already has very easy dynamic dispatching?
Would anyone be able to compare this library's design to https://github.com/google/highway?
There is also https://github.com/wingertge/macerator if you need f16 support too
Congratulations.
Nice work. Looking forward to it.
[dead]