A new instruction set for the string kernels — AVX-512 or RISC-V Vector
- Dominant language
- C++
- Stars
- 2.1k
- Forks
- 125
- Avg merge
- 5h 42m
- Merged PRs (30d)
- 136
Description
## What is missing
ripwire's text scanning runs through `src/infra/strkern.h`, one header of byte-parallel kernels with
NEON, AVX2 and scalar paths. Two kinds of hardware still fall back to the scalar twins: x86 servers
with AVX-512, and RVA23-class RISC-V machines, where the V extension is mandatory. The job is to add
one path, prove it bit-for-bit against the scalar twin on hardware CI does not have, and then
measure whether it actually helps.
## The evidence
When PR #127 landed the string kernels, warm `--pack-task` on golang/go dropped **27.6%**, with
byte-identical output. That is the size of the prize this work is chasing on two more instruction
sets.
The bar is deliberately high and honest, and a measured "no" is a welcome result: a STEP 0 profile
decides whether to start at all, parity comes before speed, and the kernel contract is already
written down in `strkern.h`, `test/verify_strkern.cpp` and `test/strkerncheck.sh`. The kit carries
verification plans for real hardware, Intel SDE, and `qemu-riscv64` at three VLENs, and states what
GitHub-hosted CI can and cannot run.
One trap is recorded from #127 and worth reading before anything else: an AVX2 capability probe
compiled to `addb` with zero ymm opcodes and reported "ok" everywhere. Let the real slice be the
probe, or disassemble it.
## Size
**Medium**, with a small first step and a large tail:
- **STEP 0 alone** — profile shares, the ceiling, possibly "stop here": small.
- **RVV find and fold kernels, or 256-bit AVX-512BW+VL**: medium.
- **512-bit blocks with 64-bit masks through the tokenizer**: large.
## Prerequisites
A C++23 toolchain and CMake, comfort with SIMD intrinsics and bit manipulation, and the ability to
run the G1 sanitizer build. For AVX-512: a Linux host whose CPU reports `avx512bw`, or an Intel 64
host that can run Intel SDE. For RVV: Ubuntu 24.04 with `g++-14-riscv64-linux-gnu` and `qemu-user`;
real RVA23 hardware is optional.
## Where to start
`prompts/help-wanted/simd-more-isas.md` is a self-contained prompt for a coding agent. It carries
the STEP 0 profile, the kernel contract, the two verification plans, the design options in order of
blast radius, the traps #127 met, the acceptance criteria, and what the implementation pull request
must report. Like every prompt in `prompts/`, **it ends by writing a plan and stopping** — a
maintainer agrees the plan before any code is written. Post that plan and its STEP 0 numbers as a
draft pull request before any kernels are written.
Comment here to claim it, naming the instruction set being taken.
Contributor guide
Research direction
Start with prompts/help-wanted/simd-more-isas.md, then read the kernel contract in src/infra/strkern.h and the checks in test/verify_strkern.cpp and test/strkerncheck.sh. Run the STEP 0 profile first and post its numbers with a plan as a draft pull request. Done means a maintainer-approved plan, bit-for-bit verification on the stated hardware or emulation path, and measured performance results.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- cli, performance
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100