rust-lang / rust-lang/rust

Poor codegen for `_mm512_permutex2var_epi8` intrinsic on avx512vbmi

Open
#156,946 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

A-codegen A-LLVM C-bug I-slow needs-triage
Dominant language
Rust
Stars
119k
Forks
16.1k
PR merge metrics
PR metrics pending

Description

_mm512_permutex2var_epi8 lowers to vpermi2w for byte shuffle representable as word shuffle. Both forms are correct but rustc picks the slower one.

This reproduces across all AVX-512 targets selected with -C target-cpu=, from skylake-avx512 to znver5.

I tried this code:

#![crate_type = "lib"]

use core::arch::x86_64::*;

#[no_mangle]
#[target_feature(enable = "avx512f,avx512bw,avx512vbmi")]
pub unsafe fn repro_byte_permute(a: __m512i, b: __m512i) -> __m512i {
    let idx: __m512i = core::mem::transmute([
        0i8, 1, 64, 65, 2, 3, 66, 67, 4, 5, 68, 69, 6, 7, 70, 71,
        8, 9, 72, 73, 10, 11, 74, 75, 12, 13, 76, 77, 14, 15, 78, 79,
        16, 17, 80, 81, 18, 19, 82, 83, 20, 21, 84, 85, 22, 23, 86, 87,
        24, 25, 88, 89, 26, 27, 90, 91, 28, 29, 92, 93, 30, 31, 94, 95,
    ]);

    _mm512_permutex2var_epi8(a, idx, b)
}

Compiled with:

rustc +1.95 --edition=2021 -O -C target-cpu=sapphirerapids --emit asm repro.rs
rustc +nightly --edition=2021 -O -C target-cpu=sapphirerapids --emit asm repro.rs

Godbolt: https://rust.godbolt.org/z/ejexqE48j

I expected to see this happen: repro_byte_permute should lower to a byte permute instruction, e.g. vpermi2b or vpermt2b, because the Rust code explicitly calls _mm512_permutex2var_epi8.

This matters for performance. With llvm-mca-14 -mcpu=sapphirerapids, two independent register-register byte permutes model as Block RThroughput: 2.0, while two independent word permutes model as Block RThroughput: 4.0:

vpermi2b %zmm2, %zmm0, %zmm1
vpermi2b %zmm3, %zmm0, %zmm4

vpermi2w %zmm2, %zmm0, %zmm1
vpermi2w %zmm3, %zmm0, %zmm4

Instead, this happened: both rustc 1.95 and nightly emit vpermi2w for the _epi8 intrinsic:

repro_byte_permute:
    vmovdqa64   (%rsi), %zmm0
    vmovdqa64   .LCPI0_0(%rip), %zmm1
    vpermi2w    (%rdx), %zmm0, %zmm1
    vmovdqa64   %zmm1, (%rdi)
    vzeroupper
    retq

The LLVM IR still contains a <64 x i8> shuffle:

%_4.i = shufflevector <64 x i8> %0, <64 x i8> %1,
  <64 x i32> <i32 0, i32 1, i32 64, i32 65, ...>

So it looks like the intrinsic is lowered to a generic byte shufflevector, then the backend recognizes that this particular byte pattern preserves 16-bit pairs and selects vpermi2w. That is functionally correct, but appears to be a worse instruction choice on targets where AVX512VBMI byte permutes are available.

Meta

rustc +1.95 --version --verbose:

rustc 1.95.0 (59807616e 2026-04-14)
binary: rustc
commit-hash: 59807616e1fa2540724bfbac14d7976d7e4a3860
commit-date: 2026-04-14
host: x86_64-unknown-linux-gnu
release: 1.95.0
LLVM version: 22.1.2

rustc +nightly --version --verbose:

rustc 1.98.0-nightly (23a3312d9 2026-05-23)
binary: rustc
commit-hash: 23a3312d92a1c4ba0373f1e25277be20ba8bb28c
commit-date: 2026-05-23
host: x86_64-unknown-linux-gnu
release: 1.98.0-nightly
LLVM version: 22.1.6

cc @folkertdev

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the issue with the provided rustc commands and inspecting the emitted LLVM IR and assembly for _mm512_permutex2var_epi8. Trace how the <64 x i8> shufflevector reaches x86 instruction selection, then verify that AVX512VBMI targets select a byte permute and that other targets remain correct.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
compilers, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
54/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.