[X86] use vdivph for i8 integer division on avx512fp16/avx10.2
- Dominant language
- LLVM
- Stars
- 40.5k
- Forks
- 18.7k
- PR merge metrics
- PR metrics pending
Description
After #205263, i8 integer division will keep vectorization by f32 division. However, on avx512fp16/avx10.2, it can be even better: use f16 division.
```c++
#include
#include
typedef uint8_t u8 [[clang::ext_vector_type(16)]];
typedef int8_t s8 [[clang::ext_vector_type(16)]];
typedef _Float16 f16 [[clang::ext_vector_type(16)]];
typedef float f32 [[clang::ext_vector_type(16)]];
u8 avx512(u8 a, u8 b){
f32 x = __builtin_convertvector(a, f32);
f32 y = __builtin_convertvector(b, f32);
return __builtin_convertvector(x/y, u8);
}
u8 avx512fp16(u8 a, u8 b){
f16 x = __builtin_convertvector(a, f16);
f16 y = __builtin_convertvector(b, f16);
return _mm256_cvtepi16_epi8(_mm256_cvttph_epu16(x/y));
}
s8 avx512(s8 a, s8 b){
f32 x = __builtin_convertvector(a, f32);
f32 y = __builtin_convertvector(b, f32);
return __builtin_convertvector(x/y, s8);
}
s8 avx512fp16(s8 a, s8 b){
f16 x = __builtin_convertvector(a, f16);
f16 y = __builtin_convertvector(b, f16);
return _mm256_cvtepi16_epi8(_mm256_cvttph_epi16(x/y));
}
```
```asm
avx512(unsigned char vector[16], unsigned char vector[16]):
vpmovzxbd zmm0, xmm0
vcvtdq2ps zmm0, zmm0
vpmovzxbd zmm1, xmm1
vcvtdq2ps zmm1, zmm1
vdivps zmm0, zmm0, zmm1
vcvttps2dq zmm0, zmm0
vpmovdb xmm0, zmm0
vzeroupper
ret
avx512fp16(unsigned char vector[16], unsigned char vector[16]):
vpmovzxbw ymm0, xmm0
vcvtw2ph ymm0, ymm0
vpmovzxbw ymm1, xmm1
vcvtw2ph ymm1, ymm1
vdivph ymm0, ymm0, ymm1
vcvttph2uw ymm0, ymm0
vpmovwb xmm0, ymm0
vzeroupper
ret
avx512(signed char vector[16], signed char vector[16]):
vpmovsxbd zmm0, xmm0
vcvtdq2ps zmm0, zmm0
vpmovsxbd zmm1, xmm1
vcvtdq2ps zmm1, zmm1
vdivps zmm0, zmm0, zmm1
vcvttps2dq zmm0, zmm0
vpmovdb xmm0, zmm0
vzeroupper
ret
avx512fp16(signed char vector[16], signed char vector[16]):
vpmovsxbw ymm0, xmm0
vcvtw2ph ymm0, ymm0
vpmovsxbw ymm1, xmm1
vcvtw2ph ymm1, ymm1
vdivph ymm0, ymm0, ymm1
vcvttph2w ymm0, ymm0
vpmovwb xmm0, ymm0
vzeroupper
ret
```
Current avx512fp16 hardware apparently internally emulate vdivph by vdivps hardware, which results in poor performance. So this method should only be used on tuning with fast vdivph (possibly zen6/novalake?).
https://uops.info/html-instr/VDIVPH_ZMM_ZMM_ZMM.html
```
Emerald Rapids
Measurements
Latencies
Latency operand 2 → 1: ≤41
Latency operand 3 → 1: ≤41
Throughput
Computed from the port usage: 4.00
Measured (loop): 32.00
Measured (unrolled): 32.00
Number of μops
Executed: 6
Retire slots: 6
Decoded (MITE): 4
Microcode Sequencer (MS): 2
Requires the complex decoder (no other instruction can be decoded with simple decoders in the same cycle)
Port usage: 4*p0+1*p06+1*p5
Alder Lake-P
Measurements
Latencies
Latency operand 2 → 1: ≤41
Latency operand 3 → 1: ≤41
Throughput
Computed from the port usage: 4.00
Measured (loop): 32.00
Measured (unrolled): 32.00
Number of μops
Executed: 6
Retire slots: 6
Decoded (MITE): 4
Microcode Sequencer (MS): 2
Requires the complex decoder (no other instruction can be decoded with simple decoders in the same cycle)
Port usage: 4*p0+1*p06+1*p5
```
https://godbolt.org/z/4zbebz84b
Contributor guide
Assessment
This issue has not been assessed yet.