[X86] Missed code-size optimization: no pass promotes VEX→EVEX when EVEX's `disp8*N` yields a shorter encoding
- Dominant language
- LLVM
- Stars
- 40.5k
- Forks
- 18.7k
- PR merge metrics
- PR metrics pending
Description
## Summary
For a memory-operand vector instruction that has both a VEX and an EVEX encoding, the EVEX form is sometimes strictly shorter than the VEX form, because EVEX supports a compressed displacement (`disp8*N`) that VEX does not. Upstream LLVM never exploits this:
`X86CompressEVEX` rewrites EVEX→VEX on the assumption that VEX is always at least as small, and there is no pass performing the reverse rewrite. On `prefer-evex-for-size` targets (currently `znver5`) this leaves a measurable amount of `.text` on the table.
All byte counts below are reproducible with `llvm-mc --show-encoding` and `llc -show-mc-encoding`; exact commands are given inline.
## Encoding background
The relevant size difference between a VEX and an EVEX encoding of the same instruction has exactly two components:
1. **Prefix length.** A VEX prefix is 2 bytes when it uses the `0xC5` (two-byte) form and 3 bytes when it uses the `0xC4` (three-byte) form. An EVEX prefix is always 4 bytes and begins with `0x62`. In isolation, EVEX is therefore 1–2 bytes larger than VEX.
2. **Displacement.** A memory operand's displacement is encoded either as `disp8` (1 signed byte, range −128..127) or `disp32` (4 bytes). VEX has only these two options, so any displacement with `|disp| > 127` costs a full `disp32`. EVEX additionally supports a *compressed* displacement, `disp8*N` (a.k.a. CDisp8): the stored signed byte is scaled by the instruction form's CD8 factor `N`. EVEX can therefore use a 1-byte displacement whenever `disp % N == 0` and `disp / N` lies in −128..127, i.e. for any `disp` in `[−128*N, 127*N]` that is a multiple of `N`.
Consequently, when `127 < |disp| <= 127*N` and `disp % N == 0`, the VEX encoding must spend a 4-byte `disp32` while the EVEX encoding spends a 1-byte `disp8*N`. The 3-byte displacement saving exceeds EVEX's 1–2-byte prefix penalty, making the EVEX encoding 1–2 bytes shorter overall. Outside that displacement window EVEX is equal or larger, so the win is specific to this range.
This is only decidable for the low 16 vector registers (`xmm0`–`xmm15`), since those are the only ones that have both a VEX and an EVEX encoding. AVX512VL is the feature that makes the EVEX encoding available on 128/256-bit operands; `znver5` has it.
## The missed case
`X86CompressEVEX` (`llvm/lib/Target/X86/X86CompressEVEX.cpp`) rewrites EVEX instructions to their VEX twins to reduce code size. Its documentation states this compression "can always reduce code size", and `CompressEVEXImpl` does not inspect the memory displacement but instead, gates only on mask/512-bit (`EVEX_K | EVEX_L2`) and broadcast/rounding (`EVEX_B`) forms. As a result it will rewrite an 8-byte EVEX instruction into a 9-byte VEX one in the `disp8*N` window described above.
The byte-level compressed-displacement predicate itself already exists and is correct: `isDispOrCDisp8` in `llvm/lib/Target/X86/MCTargetDesc/X86MCCodeEmitter.cpp` reads the per-form CD8 scale, rejects unaligned displacements, and checks `isInt<8>(disp / N)`. The gap is purely at the instruction-selection/rewrite level: nothing promotes a VEX instruction to its EVEX twin when that predicate would hold, and there is no VEX→EVEX converter in the backend at all.
## Reproducer
```
llc -mtriple=x86_64-- -mattr=+avx512vl,+avx512dq,+prefer-evex-for-size -show-mc-encoding
```
```llvm
define <4 x float> @addps_bigdisp(ptr %p, <4 x float> %x) {
%g = getelementptr i8, ptr %p, i64 2032 ; 2032 = 127 * 16 (N=16), compressible
%l = load <4 x float>, ptr %g, align 16
%a = fadd <4 x float> %x, %l
ret <4 x float> %a
}
```
- Emitted today: `vaddps 2032(%rdi), %xmm0, %xmm0` as VEX (`0xC5…`) with a 4-byte
`disp32` — 9 bytes.
- Available EVEX twin `VADDPSZ128rm`: same effective address as `disp8*N`
(`0x62…, 0x7F`, since `2032 / 16 = 127`) — 8 bytes.
The equivalence can be checked directly at the MC layer:
```
$ echo '{vex} vmovsd 0x2c8(%rsp), %xmm7' | llvm-mc --show-encoding -mattr=+avx512vl -triple x86_64
# encoding: [0xc5,0xfb,0x10,0xbc,0x24,0xc8,0x02,0x00,0x00] (9 bytes)
$ echo '{evex} vmovsd 0x2c8(%rsp), %xmm7' | llvm-mc --show-encoding -mattr=+avx512vl -triple x86_64
# encoding: [0x62,0xf1,0xff,0x08,0x10,0x7c,0x24,0x59] (8 bytes; 0x2c8/8 = 0x59)
```
The move family (`vmovaps`, `vmovsd`, …) exhibits the same 9-vs-8 difference; the point of this report is that arithmetic, FMA, scalar, and imm8-carrying memory forms that appear in the same VEX↔EVEX equivalence table are affected identically and are equally promotable.
## Scope and impact
- Affects only `prefer-evex-for-size` subtargets (currently `znver5`); default-off elsewhere.
- Pure code-size change; the emitted instruction is semantically identical (same effective address, same operation).
- The per-instruction saving is 1–2 bytes, but the pattern is common in code with large stack frames or large struct/array offsets, so the aggregate `.text` reduction is non-trivial on real programs.
Contributor guide
Assessment
This issue has not been assessed yet.