Know constant value at compile time cause slower time than now knowing.
- Dominant language
- LLVM
- Stars
- 40.5k
- Forks
- 18.7k
- PR merge metrics
- PR metrics pending
Description
### Background
I am a student and experimenting a matrix multiplication function.
As I tried doing block-tiling algorithm, and tried setting the dimension as know at compiled time and unknown at compile time, the unknown version turn out to get faster result. I inspected the assembly code and found different choices of optimizations. Intuitively, I believe that enabling compile time knowledge could at least make the code as good as before or better than before.
I am not sure if I miss any compiler flags or create a faulty benchmark code. Please let me know if I made mistakes.
*Note : I use Google Benchmark (https://github.com/google/benchmark) for benchmarking the function*
### Code
main.cpp
```cpp
#include
#include
#include
#include
#include
#include "function.h"
using namespace std;
static void escape(void *p) {
asm volatile("" : : "g"(p) : "memory");
}
static void MatMul(benchmark::State& state) {
Tensor a(dim1, dim2);
Tensor b(dim2, dim3);
Tensor c(dim1, dim3);
for(int q = 0;q < dim1 * dim2;q++) {
a[q] = std::rand();
}
for(int q = 0;q < dim2 * dim3;q++) {
b[q] = std::rand();
}
for(auto _ : state) {
MatMulPlusAB(a, b, c);
escape(b.data);
}
state.SetItemsProcessed(state.iterations());
}
BENCHMARK(MatMul);
BENCHMARK_MAIN();
```
function.h
```cpp
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include
static constexpr int BLOCK_SIZE = 64;
static constexpr int dim1 = 64;
static constexpr int dim2 = 512;
static constexpr int dim3 = 2048;
class Tensor;
class TensorView {
public:
inline TensorView();
inline TensorView(const TensorView& other);
inline TensorView(Tensor& t);
inline TensorView(float* data, int row, int col);
inline float& operator[](const int idx);
inline const float& operator[](const int idx) const;
inline TensorView& operator=(const float x);
inline TensorView& operator=(const TensorView other);
inline TensorView& operator+=(const TensorView other);
inline TensorView sliceRow(int r0,int r);
float* data;
int row;
int col;
};
class Tensor {
public:
inline Tensor();
inline Tensor(const int row, const int col);
inline ~Tensor();
inline float& operator[](const int idx);
inline const float& operator[](const int idx) const;
inline Tensor& operator=(const float x);
inline Tensor& operator=(const TensorView other);
inline Tensor& operator+=(const TensorView other);
inline TensorView sliceRow(int r0,int r);
float* data;
int row;
int col;
};
inline TensorView::TensorView() : data(nullptr), row(0), col(0) {}
inline TensorView::TensorView(const TensorView& other)
: data(other.data), row(other.row), col(other.col) {}
inline TensorView::TensorView(Tensor& t)
: data(t.data), row(t.row), col(t.col) {}
inline TensorView::TensorView(float* data, int row, int col)
: data(data), row(row), col(col) {}
inline float& TensorView::operator[](const int idx) {
return *(data + idx);
}
inline const float& TensorView::operator[](const int idx) const {
return *(data + idx);
}
inline TensorView& TensorView::operator=(const float x) {
for(int i = 0;i < row * col;i++) {
data[i] = x;
}
return *this;
}
inline TensorView& TensorView::operator=(const TensorView other) {
std::memcpy(data, other.data, sizeof(float) * row * col);
return *this;
}
inline TensorView& TensorView::operator+=(const TensorView other) {
for(int i = 0;i < row * col;i ++) {
data[i] += other[i];
}
return *this;
}
inline TensorView TensorView::sliceRow(int r0,int r) {
return TensorView(data + r0 * col, r, col);
}
inline Tensor::Tensor() : data(nullptr) {;}
inline Tensor::Tensor(const int row, const int col) : row(row), col(col) {
data = new float[row * col];
}
inline Tensor::~Tensor() {
delete[] data;
}
inline float& Tensor::operator[](const int idx) {
return *(data + idx);
}
inline const float& Tensor::operator[](const int idx) const {
return *(data + idx);
}
inline Tensor& Tensor::operator=(const float x) {
for(int i = 0;i < row * col;i++) {
data[i] = x;
}
return *this;
}
inline Tensor& Tensor::operator=(const TensorView other) {
std::memcpy(data, other.data, sizeof(float) * row * col);
return *this;
}
inline Tensor& Tensor::operator+=(const TensorView other) {
for(int i = 0;i < row * col;i ++) {
data[i] += other[i];
}
return *this;
}
inline TensorView Tensor::sliceRow(int r0,int r) {
return TensorView(data + r0 * col, r, col);
}
inline void MatMulPlusAB(TensorView A, TensorView B, TensorView C) {
// The dimension is not known at compile time
const int d1 = C.row;
const int d2 = A.col;
const int d3 = C.col;
// The dimension is known at compile time
// const int d1 = dim1;
// const int d2 = dim2;
// const int d3 = dim3;
for(int ii = 0;ii < d1;ii += BLOCK_SIZE) {
for(int jj = 0;jj < d3;jj += BLOCK_SIZE) {
for(int kk = 0;kk < d2;kk += BLOCK_SIZE) {
for(int i = 0;(i < BLOCK_SIZE) && (ii + i < d1);i++) {
for(int k = 0; (k < BLOCK_SIZE ) && (kk + k < d2);k++) {
for(int j = 0;(j < BLOCK_SIZE) && (jj + j < d3);j++) {
C[(ii + i) * d3 + (jj + j)] += A[(ii + i) * d2 + (kk + k)] * B[(kk + k) * d3 + (jj + j)];
}
}
}
}
}
}
}
```
### Running Result
Unknown at compile time
```
2025-11-11T15:44:45+07:00
Running ./main.exe
Run on (16 X 2949.21 MHz CPU s)
CPU Caches:
L1 Data 32 KiB (x8)
L1 Instruction 32 KiB (x8)
L2 Unified 512 KiB (x8)
L3 Unified 4096 KiB (x2)
Load Average: 0.54, 0.39, 0.32
---------------------------------------------------------------------
Benchmark Time CPU Iterations UserCounters...
---------------------------------------------------------------------
MatMul 3006416 ns 3005636 ns 232 items_per_second=332.708/s
```
Known at compile time
```
2025-11-11T15:46:09+07:00
Running ./main.exe
Run on (16 X 4272.86 MHz CPU s)
CPU Caches:
L1 Data 32 KiB (x8)
L1 Instruction 32 KiB (x8)
L2 Unified 512 KiB (x8)
L3 Unified 4096 KiB (x2)
Load Average: 0.65, 0.44, 0.34
---------------------------------------------------------------------
Benchmark Time CPU Iterations UserCounters...
---------------------------------------------------------------------
MatMul 6291752 ns 6290681 ns 115 items_per_second=158.965/s
```
### Inspection Result (perf report)
With the dimension unknown at compiler time
*The following codes are observed to be unrolled multiple times*
```
0.88 │640:┌─→vbroadcastss (%rbx,%rsi,4),%ymm8 ▒
0.90 │ │ inc %rsi ▒
1.49 │ │ vfmadd231ps -0xe0(%rdi),%ymm8,%ymm7 ▒
1.04 │ │ vfmadd231ps -0xc0(%rdi),%ymm8,%ymm6 ▒
1.57 │ │ vfmadd231ps -0xa0(%rdi),%ymm8,%ymm5 ▒
1.63 │ │ vfmadd231ps -0x80(%rdi),%ymm8,%ymm4 ▒
1.36 │ │ vfmadd231ps -0x60(%rdi),%ymm8,%ymm3 ◆
1.17 │ │ vfmadd231ps -0x40(%rdi),%ymm8,%ymm2 ▒
0.71 │ │ vfmadd231ps -0x20(%rdi),%ymm8,%ymm1 ▒
1.43 │ │ vfmadd231ps (%rdi),%ymm8,%ymm0 ▒
0.69 │ │ add $0x2000,%rdi ▒
0.89 │ ├──cmp $0x40,%rsi ▒
0.02 │ └──jne 640
```
With the dimension known at compile time
*The following codes are observed to be unrolled multiple times*
```
│ data16 data16 data16 data16 cs nopw 0x0(%rax,%rax,1) ▒
0.10 │ a80:┌─→mov %r13,%rax ▒
0.07 │ │ vbroadcastss 0x400(%r14,%r13,4),%ymm0 ▒
0.26 │ │ shl $0xd,%rax ▒
0.46 │ │ lea 0x200000(%rax,%rcx,1),%rbx ▒
0.39 │ │ vmovups (%rbx,%r12,4),%ymm1 ▒
0.82 │ │ vmovups 0x20(%rbx,%r12,4),%ymm2 ▒
0.53 │ │ vfmadd213ps (%r11,%r12,4),%ymm0,%ymm1 ▒
0.23 │ │ vfmadd213ps 0x20(%r11,%r12,4),%ymm0,%ymm2 ▒
0.56 │ │ vmovups %ymm1,(%r11,%r12,4) ▒
0.33 │ │ vmovups %ymm2,0x20(%r11,%r12,4) ▒
0.07 │ │ vbroadcastss 0x400(%r14,%r13,4),%ymm0 ▒
0.36 │ │ vmovups (%rbx,%r8,4),%ymm1 ▒
0.36 │ │ vmovups 0x20(%rbx,%r8,4),%ymm2 ▒
0.82 │ │ vfmadd213ps (%r11,%r8,4),%ymm0,%ymm1 ▒
0.40 │ │ vfmadd213ps 0x20(%r11,%r8,4),%ymm0,%ymm2 ▒
0.13 │ │ vmovups %ymm1,(%r11,%r8,4) ▒
0.33 │ │ vmovups %ymm2,0x20(%r11,%r8,4) ▒
0.43 │ │ vbroadcastss 0x400(%r14,%r13,4),%ymm0 ▒
0.26 │ │ vmovups (%rbx,%r10,4),%ymm1 ▒
0.27 │ │ vmovups 0x20(%rbx,%r10,4),%ymm2 ▒
0.59 │ │ vfmadd213ps (%r11,%r10,4),%ymm0,%ymm1 ▒
0.63 │ │ vfmadd213ps 0x20(%r11,%r10,4),%ymm0,%ymm2 ▒
0.36 │ │ vmovups %ymm1,(%r11,%r10,4) ▒
0.46 │ │ vmovups %ymm2,0x20(%r11,%r10,4) ◆
0.49 │ │ vbroadcastss 0x400(%r14,%r13,4),%ymm0 ▒
0.56 │ │ vmovups (%rbx,%rsi,4),%ymm1 ▒
0.23 │ │ vmovups 0x20(%rbx,%rsi,4),%ymm2 ▒
0.16 │ │ vfmadd213ps (%r11,%rsi,4),%ymm0,%ymm1 ▒
0.16 │ │ vfmadd213ps 0x20(%r11,%rsi,4),%ymm0,%ymm2 ▒
0.33 │ │ vmovups %ymm1,(%r11,%rsi,4) ▒
0.52 │ │ vmovups %ymm2,0x20(%r11,%rsi,4) ▒
0.19 │ b51:│ inc %r13 ▒
0.30 │ │ add $0x2000,%rdx ▒
0.23 │ │ cmp $0x40,%r13 ▒
│ │↑ je 9f0 ▒
0.10 │ b65:├──test %r15b,%r15b ▒
│ └──je a80
```
### Environment
**clang --version**
```
Ubuntu clang version 19.1.1 (1ubuntu1)
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/lib/llvm-19/bin
```
**Compile command**
```
clang++ ./main.cpp -std=c++17 -isystem benchmark/include -Lbenchmark/build/src -lbenchmark -lpthread -o ./main.exe -O3 -fno-omit-frame-pointer -funroll-loops -ftree-vectorize -mavx2 -msse -ffast-math -march=native -fveclib=libmvec
```
**lscpu**
```
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Address sizes: 48 bits physical, 48 bits virtual
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: AuthenticAMD
Model name: AMD Ryzen 7 4800H with Radeon Graphics
CPU family: 23
Model: 96
Thread(s) per core: 2
Core(s) per socket: 8
Socket(s): 1
Stepping: 1
Frequency boost: enabled
CPU(s) scaling MHz: 88%
CPU max MHz: 4300.0000
CPU min MHz: 400.0000
BogoMIPS: 5789.27
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_
good nopl xtopology nonstop_tsc cpuid extd_apicid aperfmperf rapl pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm
cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpb cat_l3
cdp_l3 hw_pstate ssbd mba ibrs ibpb stibp vmmcall fsgsbase bmi1 avx2 smep bmi2 cqm rdt_a rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 cqm_llc cqm
_occup_llc cqm_mbm_total cqm_mbm_local clzero irperf xsaveerptr rdpru wbnoinvd cppc arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists
pausefilter pfthreshold avic v_vmsave_vmload vgif v_spec_ctrl umip rdpid overflow_recov succor smca
Virtualization features:
Virtualization: AMD-V
Caches (sum of all):
L1d: 256 KiB (8 instances)
L1i: 256 KiB (8 instances)
L2: 4 MiB (8 instances)
L3: 8 MiB (2 instances)
NUMA:
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerabilities:
Gather data sampling: Not affected
Itlb multihit: Not affected
L1tf: Not affected
Mds: Not affected
Meltdown: Not affected
Mmio stale data: Not affected
Reg file data sampling: Not affected
Retbleed: Mitigation; untrained return thunk; SMT enabled with STIBP protection
Spec rstack overflow: Mitigation; Safe RET
Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Spectre v2: Mitigation; Retpolines; IBPB conditional; STIBP always-on; RSB filling; PBRSB-eIBRS Not affected; BHI Not affected
Srbds: Not affected
Tsx async abort: Not affected
```
**sudo dmidecode --type memory**
```
# dmidecode 3.6
Getting SMBIOS data from sysfs.
SMBIOS 3.2.0 present.
Handle 0x000A, DMI type 16, 23 bytes
Physical Memory Array
Location: System Board Or Motherboard
Use: System Memory
Error Correction Type: None
Maximum Capacity: 32 GB
Error Information Handle: 0x0009
Number Of Devices: 2
Handle 0x0011, DMI type 17, 84 bytes
Memory Device
Array Handle: 0x000A
Error Information Handle: 0x0010
Total Width: Unknown
Data Width: Unknown
Size: No Module Installed
Form Factor: Unknown
Set: None
Locator: DIMM 0
Bank Locator: P0 CHANNEL A
Type: Unknown
Type Detail: Unknown
Handle 0x0013, DMI type 17, 84 bytes
Memory Device
Array Handle: 0x000A
Error Information Handle: 0x0012
Total Width: 64 bits
Data Width: 64 bits
Size: 8 GB
Form Factor: SODIMM
Set: None
Locator: DIMM 0
Bank Locator: P0 CHANNEL B
Type: DDR4
Type Detail: Synchronous Unbuffered (Unregistered)
Speed: 3200 MT/s
Manufacturer: Micron Technology
Serial Number: 2D611821
Asset Tag: Not Specified
Part Number: 4ATF1G64HZ-3G2E2
Rank: 1
Configured Memory Speed: 3200 MT/s
Minimum Voltage: 1.2 V
Maximum Voltage: 1.2 V
Configured Voltage: 1.2 V
Memory Technology: DRAM
Memory Operating Mode Capability: Volatile memory
Firmware Version: Unknown
Module Manufacturer ID: Bank 1, Hex 0x2C
Module Product ID: Unknown
Memory Subsystem Controller Manufacturer ID: Unknown
Memory Subsystem Controller Product ID: Unknown
Non-Volatile Size: None
Volatile Size: 8 GB
Cache Size: None
Logical Size: None
```
Contributor guide
Research direction
Start with main.cpp and function.h, focusing on the MatMul benchmark and MatMulPlusAB with the two dimension definitions. Rebuild and run the Google Benchmark cases, then compare the generated code and perf output; done means explaining or reproducing the performance difference between compile-time-known and runtime dimensions.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- compilers, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100