redhat-et / redhat-et/ripwire

A new instruction set for the string kernels — AVX-512 or RISC-V Vector

Open
#161 0 comments 0 reactions 0 assignees View on GitHub
help wanted
Dominant language
C++
Stars
2.1k
Forks
125
Avg merge
5h 42m
Merged PRs (30d)
136

Description

## What is missing

ripwire's text scanning runs through `src/infra/strkern.h`, one header of byte-parallel kernels with
NEON, AVX2 and scalar paths. Two kinds of hardware still fall back to the scalar twins: x86 servers
with AVX-512, and RVA23-class RISC-V machines, where the V extension is mandatory. The job is to add
one path, prove it bit-for-bit against the scalar twin on hardware CI does not have, and then
measure whether it actually helps.

## The evidence

When PR #127 landed the string kernels, warm `--pack-task` on golang/go dropped **27.6%**, with
byte-identical output. That is the size of the prize this work is chasing on two more instruction
sets.

The bar is deliberately high and honest, and a measured "no" is a welcome result: a STEP 0 profile
decides whether to start at all, parity comes before speed, and the kernel contract is already
written down in `strkern.h`, `test/verify_strkern.cpp` and `test/strkerncheck.sh`. The kit carries
verification plans for real hardware, Intel SDE, and `qemu-riscv64` at three VLENs, and states what
GitHub-hosted CI can and cannot run.

One trap is recorded from #127 and worth reading before anything else: an AVX2 capability probe
compiled to `addb` with zero ymm opcodes and reported "ok" everywhere. Let the real slice be the
probe, or disassemble it.

## Size

**Medium**, with a small first step and a large tail:

- **STEP 0 alone** — profile shares, the ceiling, possibly "stop here": small.
- **RVV find and fold kernels, or 256-bit AVX-512BW+VL**: medium.
- **512-bit blocks with 64-bit masks through the tokenizer**: large.

## Prerequisites

A C++23 toolchain and CMake, comfort with SIMD intrinsics and bit manipulation, and the ability to
run the G1 sanitizer build. For AVX-512: a Linux host whose CPU reports `avx512bw`, or an Intel 64
host that can run Intel SDE. For RVV: Ubuntu 24.04 with `g++-14-riscv64-linux-gnu` and `qemu-user`;
real RVA23 hardware is optional.

## Where to start

`prompts/help-wanted/simd-more-isas.md` is a self-contained prompt for a coding agent. It carries
the STEP 0 profile, the kernel contract, the two verification plans, the design options in order of
blast radius, the traps #127 met, the acceptance criteria, and what the implementation pull request
must report. Like every prompt in `prompts/`, **it ends by writing a plan and stopping** — a
maintainer agrees the plan before any code is written. Post that plan and its STEP 0 numbers as a
draft pull request before any kernels are written.

Comment here to claim it, naming the instruction set being taken.

Contributor guide

Open the contributing guide

Research direction

Start with prompts/help-wanted/simd-more-isas.md, then read the kernel contract in src/infra/strkern.h and the checks in test/verify_strkern.cpp and test/strkerncheck.sh. Run the STEP 0 profile first and post its numbers with a plan as a draft pull request. Done means a maintainer-approved plan, bit-for-bit verification on the stated hardware or emulation path, and measured performance results.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
cli, performance
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.