llvm / llvm/llvm-project

[X86] use vdivph for i8 integer division on avx512fp16/avx10.2

Open
#214,431 2 comments 0 reactions 1 assignee Claimed by @AryanBhirud View on GitHub
backend:X86 missed-optimization
Dominant language
LLVM
Stars
40.5k
Forks
18.7k
PR merge metrics
PR metrics pending

Description

After #205263, i8 integer division will keep vectorization by f32 division. However, on avx512fp16/avx10.2, it can be even better: use f16 division.

```c++
#include
#include

typedef uint8_t u8 [[clang::ext_vector_type(16)]];
typedef int8_t s8 [[clang::ext_vector_type(16)]];
typedef _Float16 f16 [[clang::ext_vector_type(16)]];
typedef float f32 [[clang::ext_vector_type(16)]];

u8 avx512(u8 a, u8 b){
f32 x = __builtin_convertvector(a, f32);
f32 y = __builtin_convertvector(b, f32);
return __builtin_convertvector(x/y, u8);
}

u8 avx512fp16(u8 a, u8 b){
f16 x = __builtin_convertvector(a, f16);
f16 y = __builtin_convertvector(b, f16);
return _mm256_cvtepi16_epi8(_mm256_cvttph_epu16(x/y));
}

s8 avx512(s8 a, s8 b){
f32 x = __builtin_convertvector(a, f32);
f32 y = __builtin_convertvector(b, f32);
return __builtin_convertvector(x/y, s8);
}

s8 avx512fp16(s8 a, s8 b){
f16 x = __builtin_convertvector(a, f16);
f16 y = __builtin_convertvector(b, f16);
return _mm256_cvtepi16_epi8(_mm256_cvttph_epi16(x/y));
}
```
```asm
avx512(unsigned char vector[16], unsigned char vector[16]):
vpmovzxbd zmm0, xmm0
vcvtdq2ps zmm0, zmm0
vpmovzxbd zmm1, xmm1
vcvtdq2ps zmm1, zmm1
vdivps zmm0, zmm0, zmm1
vcvttps2dq zmm0, zmm0
vpmovdb xmm0, zmm0
vzeroupper
ret

avx512fp16(unsigned char vector[16], unsigned char vector[16]):
vpmovzxbw ymm0, xmm0
vcvtw2ph ymm0, ymm0
vpmovzxbw ymm1, xmm1
vcvtw2ph ymm1, ymm1
vdivph ymm0, ymm0, ymm1
vcvttph2uw ymm0, ymm0
vpmovwb xmm0, ymm0
vzeroupper
ret

avx512(signed char vector[16], signed char vector[16]):
vpmovsxbd zmm0, xmm0
vcvtdq2ps zmm0, zmm0
vpmovsxbd zmm1, xmm1
vcvtdq2ps zmm1, zmm1
vdivps zmm0, zmm0, zmm1
vcvttps2dq zmm0, zmm0
vpmovdb xmm0, zmm0
vzeroupper
ret

avx512fp16(signed char vector[16], signed char vector[16]):
vpmovsxbw ymm0, xmm0
vcvtw2ph ymm0, ymm0
vpmovsxbw ymm1, xmm1
vcvtw2ph ymm1, ymm1
vdivph ymm0, ymm0, ymm1
vcvttph2w ymm0, ymm0
vpmovwb xmm0, ymm0
vzeroupper
ret
```

Current avx512fp16 hardware apparently internally emulate vdivph by vdivps hardware, which results in poor performance. So this method should only be used on tuning with fast vdivph (possibly zen6/novalake?).

https://uops.info/html-instr/VDIVPH_ZMM_ZMM_ZMM.html

```
Emerald Rapids
Measurements
Latencies
Latency operand 2 → 1: ≤41
Latency operand 3 → 1: ≤41
Throughput
Computed from the port usage: 4.00
Measured (loop): 32.00
Measured (unrolled): 32.00
Number of μops
Executed: 6
Retire slots: 6
Decoded (MITE): 4
Microcode Sequencer (MS): 2
Requires the complex decoder (no other instruction can be decoded with simple decoders in the same cycle)
Port usage: 4*p0+1*p06+1*p5
Alder Lake-P
Measurements
Latencies
Latency operand 2 → 1: ≤41
Latency operand 3 → 1: ≤41
Throughput
Computed from the port usage: 4.00
Measured (loop): 32.00
Measured (unrolled): 32.00
Number of μops
Executed: 6
Retire slots: 6
Decoded (MITE): 4
Microcode Sequencer (MS): 2
Requires the complex decoder (no other instruction can be decoded with simple decoders in the same cycle)
Port usage: 4*p0+1*p06+1*p5
```
https://godbolt.org/z/4zbebz84b

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.