Poor codegen for `_mm512_permutex2var_epi8` intrinsic on avx512vbmi
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 119k
- Forks
- 16.1k
- PR merge metrics
- PR metrics pending
Description
_mm512_permutex2var_epi8 lowers to vpermi2w for byte shuffle representable as word shuffle. Both forms are correct but rustc picks the slower one.
This reproduces across all AVX-512 targets selected with -C target-cpu=, from skylake-avx512 to znver5.
I tried this code:
#![crate_type = "lib"]
use core::arch::x86_64::*;
#[no_mangle]
#[target_feature(enable = "avx512f,avx512bw,avx512vbmi")]
pub unsafe fn repro_byte_permute(a: __m512i, b: __m512i) -> __m512i {
let idx: __m512i = core::mem::transmute([
0i8, 1, 64, 65, 2, 3, 66, 67, 4, 5, 68, 69, 6, 7, 70, 71,
8, 9, 72, 73, 10, 11, 74, 75, 12, 13, 76, 77, 14, 15, 78, 79,
16, 17, 80, 81, 18, 19, 82, 83, 20, 21, 84, 85, 22, 23, 86, 87,
24, 25, 88, 89, 26, 27, 90, 91, 28, 29, 92, 93, 30, 31, 94, 95,
]);
_mm512_permutex2var_epi8(a, idx, b)
}
Compiled with:
rustc +1.95 --edition=2021 -O -C target-cpu=sapphirerapids --emit asm repro.rs
rustc +nightly --edition=2021 -O -C target-cpu=sapphirerapids --emit asm repro.rs
Godbolt: https://rust.godbolt.org/z/ejexqE48j
I expected to see this happen: repro_byte_permute should lower to a byte permute instruction, e.g. vpermi2b or vpermt2b, because the Rust code explicitly calls _mm512_permutex2var_epi8.
This matters for performance. With llvm-mca-14 -mcpu=sapphirerapids, two independent register-register byte permutes model as Block RThroughput: 2.0, while two independent word permutes model as Block RThroughput: 4.0:
vpermi2b %zmm2, %zmm0, %zmm1
vpermi2b %zmm3, %zmm0, %zmm4
vpermi2w %zmm2, %zmm0, %zmm1
vpermi2w %zmm3, %zmm0, %zmm4
Instead, this happened: both rustc 1.95 and nightly emit vpermi2w for the _epi8 intrinsic:
repro_byte_permute:
vmovdqa64 (%rsi), %zmm0
vmovdqa64 .LCPI0_0(%rip), %zmm1
vpermi2w (%rdx), %zmm0, %zmm1
vmovdqa64 %zmm1, (%rdi)
vzeroupper
retq
The LLVM IR still contains a <64 x i8> shuffle:
%_4.i = shufflevector <64 x i8> %0, <64 x i8> %1,
<64 x i32> <i32 0, i32 1, i32 64, i32 65, ...>
So it looks like the intrinsic is lowered to a generic byte shufflevector, then the backend recognizes that this particular byte pattern preserves 16-bit pairs and selects vpermi2w. That is functionally correct, but appears to be a worse instruction choice on targets where AVX512VBMI byte permutes are available.
Meta
rustc +1.95 --version --verbose:
rustc 1.95.0 (59807616e 2026-04-14)
binary: rustc
commit-hash: 59807616e1fa2540724bfbac14d7976d7e4a3860
commit-date: 2026-04-14
host: x86_64-unknown-linux-gnu
release: 1.95.0
LLVM version: 22.1.2
rustc +nightly --version --verbose:
rustc 1.98.0-nightly (23a3312d9 2026-05-23)
binary: rustc
commit-hash: 23a3312d92a1c4ba0373f1e25277be20ba8bb28c
commit-date: 2026-05-23
host: x86_64-unknown-linux-gnu
release: 1.98.0-nightly
LLVM version: 22.1.6
cc @folkertdev
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the issue with the provided rustc commands and inspecting the emitted LLVM IR and assembly for _mm512_permutex2var_epi8. Trace how the <64 x i8> shufflevector reaches x86 instruction selection, then verify that AVX512VBMI targets select a byte permute and that other targets remain correct.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- compilers, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 54/100