Proposal: opt-in benchmarks/ harness for per-query scan throughput (ThreadPool + SIMD dispatch)
- Dominant language
- C++
- Stars
- 15.9k
- Forks
- 998
- Avg merge
- 4d 7h
- Merged PRs (30d)
- 34
Description
## Proposal: opt-in `benchmarks/` harness for zvec's own primitives
Before opening a PR I'd like to check whether you'd welcome a small,
**opt-in** benchmark harness under a new top-level `benchmarks/` directory. I
have a branch ready and would only send it if this is something you want
upstream — happy to drop it otherwise.
### What it is
A single micro-benchmark, `distance_scan_bench`, that measures flat-scan
throughput **the way zvec actually scales it** — and nothing more:
- parallelism is applied **per query** across `ailego::ThreadPool`;
- each individual query is a **serial** scan that calls zvec's
**SIMD-dispatched** `Distance::SquaredEuclidean` (runtime `CpuFeatures`
dispatch: SSE / AVX / AVX512 / NEON).
No OpenMP, no hand-written intrinsics, no kernel changes. It uses exactly the
two primitives zvec already relies on. This is deliberately aligned with the
architecture (`ENABLE_OPENMP` off, hot kernels hand-vectorized, throughput from
distributing *queries* over the pool while the single-query scan stays serial)
rather than cutting across it.
```mermaid
flowchart TD
subgraph POOL["ailego::ThreadPool — parallel ACROSS queries"]
direction LR
q0["query 0
serial scan"]:::par
q1["query 1
serial scan"]:::par
qn["query N-1
serial scan"]:::par
end
POOL --> K
subgraph K["each scan: one query vs all DB vectors"]
direction TB
loop["for i in 0..num_db (serial)"]:::seq
kern["Distance::SquaredEuclidean(db_i, q, dim)
SIMD-dispatched: SSE / AVX / AVX512 / NEON"]:::simd
loop --> kern
end
classDef par fill:#1f7a1f,color:#fff,stroke:#0a0,stroke-width:2px
classDef seq fill:#2b2b2b,color:#fff,stroke:#888
classDef simd fill:#1f3a7a,color:#fff,stroke:#46f,stroke-width:2px
```
The pool parallelizes the outer axis (queries); the SIMD kernel vectorizes the
inner axis (one pair's reduction). They compose and neither touches the other's
loop, so the parallel result is **bit-identical** to serial — each query writes a
disjoint output row and the per-pair reduction order is unchanged. The benchmark
self-validates this on every run (and is registered as a `ctest`), exiting
non-zero on any mismatch.
### Zero impact when off
- New CMake option `BUILD_BENCHMARKS`, **default `OFF`** — the target does not
exist unless explicitly enabled, so the default build and CI matrix are
untouched.
- Links only `zvec_ailego`; adds no third-party dependency.
```bash
cmake -S . -B build -DBUILD_BENCHMARKS=ON -DCMAKE_BUILD_TYPE=Release
cmake --build build --target distance_scan_bench
./build/bin/distance_scan_bench # default 4096 256 128
ctest -R distance_scan_bench # self-validation
```
### Measured (illustrative)
`num_db=8192, num_query=256, dim=128`, Apple Silicon (10 cores), kernel SIMD
path **NEON**, `-O3`, best-of-5; serial baseline `0.023 s`:
| Threads | Speedup | max_abs_diff vs serial |
|--------:|--------:|-----------------------:|
| 1 | 1.00× | 0.000e+00 |
| 2 | 1.91× | 0.000e+00 |
| 4 | 3.76× | 0.000e+00 |
| 8 | 4.31× | 0.000e+00 |
```mermaid
xychart-beta
title "Per-query thread-pool scaling (NEON) — bar = measured, line = ideal"
x-axis "Threads" [1, 2, 4, 8]
y-axis "Speedup x" 0 --> 8
bar [1.00, 1.91, 3.76, 4.31]
line [1, 2, 4, 8]
```
Near-linear to the physical-core count, then flattens as the scan becomes
memory-bandwidth-bound. Inputs are synthetic and deterministic (no RNG);
correctness is established by bit-for-bit comparison against the serial
reference, not a data cohort.
### Questions for maintainers
1. Is a top-level `benchmarks/` directory welcome, or would you prefer a
different location / naming?
2. Is the opt-in `BUILD_BENCHMARKS=OFF` model acceptable, or do you have an
existing convention for performance tooling I should follow?
3. Any scope preferences (e.g. keep it to this one scan, or none at all)?
If this is of interest I'll open the PR. This follows the same
architecture-alignment approach as the rotator fix in #534.
---
*The "parallelize queries, never the reduction" schedule was derived and proven
legal (no carried dependence) with a polyhedral auto-scheduling pass —
[cluster_compilot](https://github.com/cluster2600/cluster_compilot), an
implementation of Agentic Auto-Scheduling (arXiv:2511.00592). The benchmark
implements that schedule with zvec's existing `ThreadPool` + SIMD-dispatched
kernels; it proposes no kernel change.*
Contributor guide
Assessment
This issue has not been assessed yet.