Add widen/narrow ops for all integers and floats

[Shnatsel]

We only had widen/narrow between u8 and u16 before. This PR removes that and builds a generic widen/narrow from the ground up, using the API from #127. It covers all integer and float types.

Implementations for floats, and for integers on AVX-512, NEON and WASM are a breeze. Very straightforward, love them.

SSE4.2 and AVX2 only have saturating conversions but not truncating ones for integers, so truncating ones have to be emulated on top of saturating ones. There are also no conversions to/from 64-bit integers, so those have to be emulated too.

I included saturating conversions in addition to truncating ones in the API, since all hardware has them natively, even older x86. This did end up adding a bit of complexity because 64-bit saturating casts have to be emulated on AVX2 and earlier. I think it's still worth keeping because they don't add that much. All the other complexity (truncating conversion emulation on AVX2 and earlier) is unavoidable. I've axed SSE2-specific kernels since autovectorization works okay for them, and I'm not willing to complicate this even further.

Since this is still quite a lot of complexity due to all the x86 emulation, I've added extensive tests on concrete values as well as a random test under #[ignore] to be run in release mode used to validate it, same as for other non-trivial ops like #276

Performance-wise rust-lang/rust#159464 bites us here (regression in Rust 1.96, 1.95 and earlier work fine) but it's not so bad that scalar wins, so nothing we can do about it right now (other than inline assembly but i'm not willing to go there). This is expected to be fixed in the upgrade to LLVM 23 which is coming Soon™. In the meantime this PR is tuned for 1.95 on llvm-mca.

Generic API considerations

There's an API decision on how to expose these ops to generic code, specifically the int/float distinction. It's a tension between flexibility and brevity.

In this PR I implemented both regular and saturating for floats and they just do the usual rounding, same as as casts. This abstracts over vector type if all you want is widen/narrow. However, if you want to operate on the widened vector afterwards, you need an additional generic bound, e.g. V::Widened: SimdInt<S>:

Snippet with an additional bound

use fearless_simd::{SimdFrom, prelude::*, u8x16};

#[inline(always)]
fn add_widened_halves<S, V>(value: V) -> V::Widened
where
    S: Simd,
    V: SimdInt<S> + SimdWiden<S>,
    V::Widened: SimdInt<S>,
{
    let (low, high) = value.widen();
    low + high
}

#[inline(always)]
fn fixed_width<S: Simd>(simd: S) -> [u16; 8] {
    let input = u8x16::simd_from(
        simd,
        [1, 2, 3, 4, 5, 6, 7, 8, 10, 20, 30, 40, 50, 60, 70, 80],
    );

    add_widened_halves(input).into()
}

#[inline(always)]
fn native_width<S: Simd>(simd: S) -> u16 {
    let input = S::u8s::from_fn(simd, |i| i as u8 + 1);
    add_widened_halves(input)[0]
}

This is not a problem on one function, but this extra bound is infectious, so all callers have to require it too until a concrete type is reached. So you have SimdWiden<S> + V::Widened: SimdInt<S> propagated all the way up the call stack.

Alternatively, one could argue floats really don't have the saturating variant and maybe they should only implement narrow but not saturating_narrow. So we could make two traits, SimdNarrowInt<S> and SimdNarrowFloat<S>. So instead of sometimes carrying the SimdNarrow<S> + V::Narrowed: SimdInt<S> bound your entire call stack you only carry SimdNarrowInt<S>, one having saturating_* and the other not, at the expense of not being able to abstract over ints and floats in widen/narrow ops anymore. Abstracting over ints and floats isn't that useful right now since SimdBase doesn't have anything on it, but this is something we can and probably should change now that masks aren't SimdBase, see #301.

Or we could have both SimdWiden and SimdWidenInt/SimdWidenFloat, with the current behavior and SimdIntWiden/SimdFloatWiden being shorthands to avoid carrying extra bounds - at the expense of more than one trait providing narrow and widen for each op, and that being potentially confusing.

Part of #297

Supersedes #127

[danderson]

Nice! I tried doing this a few days ago, but I got a headache trying to navigate the AVX2 mess 😂 Well done for finding a way through it!

[LaurenzV]

Overall, I agree with the direction. I don't have the time right now to think whether the generic Narrow/Widen traits express this in the best way, so I'll leave that up to you.

However, two points:

• As mentioned in the comment, I'd like to see the tests reworked. My personal preference would be to keep separate widen and narrow test modules, and just, as we've done in many other cases, create one method for each vector type (+ variation of saturating or not), and simply running it with an input vector and comparing the result against an output vector (of course fine if AI-generated). This way, it's much easier to parse. Or am I missing something why the tests were written in the way they are right now? I don't think there is much point in having roundtrip tests for those, but if you want to keep the randomized test or 1-2 other tests in widen_narrow for round trips, that's fine for me, but I don't think it's necessary. However, the core tests should be simple and easily readable, if possible.
• Do you happen to have a vello branch for this? We make quite extensive use of these instructions, I think, so it would be good to benchmark.

[LaurenzV]

Is there any guidance we can give somewhere if we don't care about either (because we have no overflows) and we just want the fastest? Or does it depend on the arch?

[Shnatsel]

It's late but I wanted to reply today, so here's Codex's overview immediately, I hope that's okay. I will see if I can work a more clear, manually written version into the docs.

Conversion	Current practical preference
Signed 16→8 or 32→16	saturating_narrow is often faster on x86 and Wasm because they have native signed saturating-pack instructions. Usually equivalent on NEON/AVX-512.
Unsigned 16→8 or 32→16	narrow is generally equal or faster; some backends need extra work to implement unsigned saturation.
Signed or unsigned 64→32	Prefer narrow. Saturation is substantially more expensive on SSE/AVX2 and Wasm, while NEON and AVX-512 have native instructions for both.
Float 64→32	No difference: saturating_narrow forwards to narrow.

[Shnatsel]

I've documented it, but it's complex enough that I think we should just add a third method that picks the right thing instead of documenting the complexity of it.

[Shnatsel]

Thank you!

Is the generator structure okay? I find the single toplevel match structure the most readable for these complex functions with many combinations of instruction sets, widths and types, which is why I also used it in #310

I think it's an improvement on what we had before and it's the best way to tame this complexity I've found so far.

[LaurenzV]

You mean handle_widen and handle_narrow? Looks fine to me I would say, I haven't double-checked all the logic but the structure seems okay.

[Shnatsel]

Yes, those. Good to know, thanks!

I wanted to make sure those can be understood by someone other than the person who wrote them. It can be hard to judge after messing with the code so much.

[Shnatsel]

@danderson any thoughts on the generic API design for this now that #301 is merged?

[Shnatsel]

@LaurenzV I believe I've addressed everything, and added relaxed_narrow in lieu of lengthy documentation on what to pick. PTAL.

[LaurenzV]

Thanks, not sure if I’ll get to it today, but hopefully tomorrow!

[LaurenzV]

Tests looking much better now, adding a relaxed variant also seems reasonable to me. Haven't checked whether anything else changed now, thanks!