magetypes: finish the native-codegen restoration — WASM pixel pack, W512 block ops, f64→int, and AVX-512's 18 missing backends
- Dominant language
- Rust
- Stars
- 12
- Forks
- 2
- Avg merge
- 13h 55m
- Merged PRs (30d)
- 28
Description
Supersedes #60, which is closed. Four of its seven boxes shipped, one was silently broken on AVX-512 until 3999a29d, and what is left is no longer one topic — it is three unfinished backends plus a design question that #60 framed badly. All claims below were re-measured against `main` on 2026-09-08 (`cargo asm` / `wasm-tools print`, release, no `-Ctarget-cpu=native`).
## Already shipped — nothing to do
- **`transpose_8x8` on x86 V3** — native `unpcklo/hi_ps` + `shuffle_ps` + `permute2f128_ps` (`magetypes/src/simd/impls/x86_v3.rs:618`, 398660da).
- **`to_u8` / `store_rgba` on x86 + ARM** — `to_u8_bytes` / `store_rgba_bytes` backend overrides: x86 `cvtps`+`packssdw`+`packuswb`, NEON `FCVTNS`+`SQXTN`+`SQXTUN` and the shift-or pack (398660da).
- **pack** — shipped under a different name as `narrow_saturating_i8`/`_u8` on `i16xN` and `narrow_saturating_i16`/`_u16` on `i32xN`, at 128/256/512, native on V3/V4/NEON/WASM. Typed on the signed source only, deliberately (#74, d78b118c).
- **extend** — shipped as `widen_low()` / `widen_high()` for `u8->u16`, `i8->i16`, `u16->u32`, `i16->i32` at all three widths (#74).
- **The AVX-512 delegation gap** — fixed in 3999a29d, guarded by a new `cargo xtask validate` check. Detail in the [#60 audit](https://github.com/imazen/archmage/issues/60#issuecomment-5593977213).
## 1. WASM `to_u8` / `store_rgba` are still scalar
`magetypes/src/simd/impls/wasm128.rs` has no `to_u8_bytes` / `store_rgba_bytes` override, so both take the trait's scalar default — the per-lane `nostd_math::roundevenf(..).clamp(0.0, 255.0) as u8`.
Measured: built a probe for `wasm32-wasip1` with `-Ctarget-feature=+simd128` and disassembled it. The module contains **zero** `f32x4.nearest`, `i32x4.trunc_sat_*`, `i16x8.narrow_*` or `i8x16.narrow_*` instructions. Its only SIMD ops are 5x `v128.load` + 5x `v128.store`, spilling the vector to linear memory, and `f32x4::to_u8` is an out-of-line `call` to `core::array::try_from_fn::<..Wasm128Token as F32x4Backend>::to_u8_bytes..>`. `store_4_rgba_u8` calls that helper once per channel. Same failure shape as the AVX-512 regression 3999a29d just fixed.
The retired concrete WASM type had the native sequence, and it ports directly — `d79b3e06:magetypes/src/simd/generated/wasm/w128.rs:1278`:
```rust
let i32s = i32x4_trunc_sat_f32x4(f32x4_nearest(self.0));
let i16s = i16x8_narrow_i32x4(i32s, i32s);
let u8s = u8x16_narrow_i16x8(i16s, i16s);
```
plus the `R | G<<8 | B<<16 | A<<24` shift-or pack for `store_rgba`, which is the same shape NEON already uses.
- [ ] `to_u8_bytes` override on `Wasm128Token` for `F32x4Backend` and `F32x8Backend`
- [ ] `store_rgba_bytes` override for both widths
- [ ] Pin it — the wasm probe above is a two-minute check and there is no asm gate on wasm today
## 2. W512 `f32x16` has no block ops at all
Not just the two the old issue named. `F32x16Backend` declares no `to_u8_bytes`, no `store_rgba_bytes`, no transpose; the generic `f32x16` exposes none of `transpose_8x8` / `transpose_8x8_copy` / `store_8x8` / `load_8x8` / `to_u8` / `store_rgba`; and `xtask/src/simd_types/generic_gen/block_ops.rs` still stops at W256 (`block_ops_*.rs` exists for f32x4/f32x8/f64x2/f64x4/i32x4/i32x8/i8x16/u32x4 and nothing wider).
The transpose is a genuine regression, not a gap that was always there: the retired concrete `f32x16` had a native AVX-512 `transpose_8x8` / `_copy` / `store_8x8` built from two `_mm512_setr_epi32` index vectors and permute2var (`d79b3e06:magetypes/src/simd/generated/x86/w512.rs:1151`). `to_u8` / `store_rgba` never existed at 512-bit, so those are new work if wanted.
Worth deciding before implementing: the old concrete signature took 8 `f32x16` vectors and used only the low 8 lanes of each, wasting half of every register. A W512 transpose that is worth having probably transposes two 8x8 blocks at once, or is 16x16. Do not port the old shape unexamined.
- [ ] Decide the W512 transpose shape (two parallel 8x8 vs 16x16 vs port-as-was)
- [ ] Extend `generic_gen/block_ops.rs` past W256
- [ ] Native `X64V4Token` override; decide whether `to_u8`/`store_rgba` are wanted at 512
## 3. The generic `f64` types have no float-to-int conversion
`f64x2_impl.rs` exposes only `from_array` / `to_array` / `from_repr` / `from_m128d`. There is no `to_i32x4_low`, and no `to_i32`/`to_i64` of any kind on `f64x2` / `f64x4` / `f64x8`. The concrete NEON form was `vcvtq_s64_f64` + `vmovn_s64` + `vcombine_s32`.
Whatever lands here should follow the #80 outcome for `f32`: a natural-per-backend op with the divergence documented, plus a `_saturating` variant contracted as Rust `as` semantics on every backend. Do not ship only the bare op.
- [ ] `f64 -> i32` conversion on the backend traits, native on x86/NEON/WASM
- [ ] Saturating variant with the uniform contract, pinned like `convert_saturating.rs`
## 4. AVX-512 tokens implement 12 of the 30 backend traits
This replaces #60's `_fast` bullet. That bullet asked for `min_fast`/`max_fast`/`abs_fast` on `i64x2`/`i64x4`/`u64x2`/`u64x4` taking a second `X64V4Token` parameter, so a V4 holder could reach `_mm_min_epi64`. The extra parameter is only necessary because **`X64V4Token` has no `I64x2Backend` at all**.
It implements exactly the 10 W512 traits, plus `F32x4Backend` / `F32x8Backend` through the hand-written delegation in `impls/x86_v4_f32_delegated.rs`. It is missing 18 of the 30 that `X64V3Token` implements:
```
F64x2Backend F64x4Backend
I8x16Backend I8x32Backend U8x16Backend U8x32Backend
I16x8Backend I16x16Backend U16x8Backend U16x16Backend
I32x4Backend I32x8Backend U32x4Backend U32x8Backend
I64x2Backend I64x4Backend U64x2Backend U64x4Backend
```
So `i64x2`, `i32x8` and `f64x2` do not compile. Confirmed: `the trait bound X64V4Token: I32x8Backend is not satisfied`. `X64V4xToken` and `Avx512Fp16Token` are in the same position.
That is the thing to fix. Extend the delegation to the other 18 (V3 is a strict subset of V4, so the existing safety argument in that file's header carries verbatim), then override the handful of methods where AVX512VL genuinely wins at 128/256-bit — `_mm_min_epi64`, `_mm_max_epi64`, `_mm_abs_epi64`, `_mm_srai_epi64` and their 256-bit forms, which are exactly the ops `_fast` was invented for. Result: `i64x2::min()` is native with no second token, no `_fast` suffix, and no cross-arch parity hole.
Two things to sort out first:
- The delegation is ~450 hand-written lines for two traits. Eighteen more written by hand is not reasonable — this should be generated, which also makes the `xtask validate` completeness check redundant. `x86_v4_f32_delegated.rs` is currently the one backend impl the generator does not emit (see CLAUDE.md).
- Decide whether a V4 token should silently get V3-width code, or whether callers wanting 128/256-bit under AVX-512 should be routed through `token.v3()`. The current half-and-half — f32 works, integers do not compile — is the worst of both.
- [ ] Decide: generate the delegation, or route V4 holders through `.v3()` explicitly
- [ ] If generating: cover all 30, with AVX512VL overrides where they win
- [ ] Only then revisit whether anything still needs a `_fast` shape (expected: nothing)
## 5. Housekeeping: dead parity-allowlist entries
`xtask/src/main.rs:1703-1708` allowlists these as known parity gaps:
```
i64x2::abs_fast / min_fast / max_fast — missing from: ARM, WASM
u64x2::min_fast / max_fast — missing from: ARM, WASM
```
No such methods exist anywhere in the tree. The entries never match, so `cargo xtask parity` is quietly carrying five lies. Delete them when item 4 is settled.
- [ ] Remove the five `_fast` entries
## Verified non-issues — do not reopen these
Carried forward from #60's audit, still true:
- `transpose_4x4`, `interleave_lo` / `interleave_hi` / `interleave` — LLVM recovers the array-roundtrip default to native shuffles on both x86 (`vunpcklps`/`vmovlhps`) and NEON (`zip1`/`zip2`/`trn2`/`uzp2`; `interleave_lo` fuses to `st2`). No override needed.
- **ARM `transpose_8x8`** — the generic default pattern-matches to mostly-native zip/trn on NEON. This is x86-only work, and x86 is done.
- W512 integer shifts — the variable-count source folds to `vpslld $imm8`.
- Core arithmetic — equal or better everywhere after the retirement; some ARM reductions improved.
## How to check any of this
The x86 probe is a scratch crate with `#[inline(never)]` wrappers around the generic entry points, built `--release` against a path dep, read with `cargo asm --lib`. When a V4 path is correct it does not merely match V3 — the linker folds the two into an alias (`to_u8_v4 = to_u8_v3`), which is a cleaner pass/fail signal than counting instructions. The wasm equivalent is the same crate for `wasm32-wasip1` with `-Ctarget-feature=+simd128`, read with `wasm-tools print`, grepping for the native op names. Neither is committed; `scripts/verify-asm.sh` covers loads/stores and `concat_shift` but none of the block ops, which is its own gap.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.