Correctness fuzzers — graph invariants, rank mass, input-order invariance, cache round-trips, census conservation
- Dominant language
- C++
- Stars
- 2.1k
- Forks
- 125
- Avg merge
- 5h 42m
- Merged PRs (30d)
- 136
Description
## What is missing
ripwire's fuzzing today proves that a tree-sitter grammar does not crash on hostile bytes. It never
runs ripwire's own resolver, call graph, ranking or cache. But the failures that matter most here do
not crash: a wrong edge, a reordered map, or one stale cached parse all produce an answer an agent
will believe. This project's contracts are already mechanical properties — byte-identical output,
warm equal to cold, a well-formed CSR, every call accounted for — which makes them ideal fuzz
oracles.
## The evidence
Gaps named in the kit, each verified on `main`:
- `test/fuzz/fuzz_ingest.cpp` is crash-only, and exercises the **vendored parser**, not ripwire's
code.
- `test/fuzz/run.sh` sweeps **15 of the 21** registered fuzz targets; `toml`, `yaml`, `csharp`, `c`,
`php` and `lua` have targets and seeds but are never swept, and `test/g1configcheck.sh` does not
check coverage.
- No gate and no CI step builds the `ripwire_test_csr` or `ripwire_test_pagerank` property tests,
although `CONTRIBUTING.md` §2 lists "the CSR property test fails" as an automatic rejection.
- `VERIFY` compiles to an optimizer assumption under `NDEBUG`, so a Release build cannot report a
violated invariant.
- CI's determinism gate diffs two runs of the **same** argv; nothing varies discovery order or parse
worker count, and neither is controllable today.
## Size
**Medium per oracle**, and there are five. Input-order invariance is the largest, because it needs a
test-only hook that must be proven inert when unset. Graph invariants (a random `IngestResult` fed
into `buildGraph`, no parser involved) and census conservation are the smallest starts. One oracle
per pull request, each with a gate observed red first and a committed replay corpus.
## Prerequisites
A Clang toolchain that ships the libFuzzer runtime (an LLVM release, or Linux); CMake refuses
`RIPWIRE_FUZZ=ON` without it. Graph-level and whole-binary oracles can also be built as
deterministic generators plus gate scripts, which need no libFuzzer at all. Census conservation
reads the dispositions PR #136 added, which is satisfied on current `main`.
## Where to start
`prompts/help-wanted/correctness-fuzzers.md` is a self-contained prompt for a coding agent. It
carries what exists with file pointers, the five oracles with precise definitions, how to build and
replay, the design constraints (determinism of the fuzzer itself, input-count-bounded CI, no timing
gates), the acceptance criteria, the known traps, and what each pull request should contain. Like
every prompt in `prompts/`, **it ends by writing a plan and stopping** — a maintainer agrees the
plan before any code is written.
A related kit, `prompts/help-wanted/graph-unit-tests.md`, covers the hand-checked unit tests and
asks for the same CI gate.
Comment here to claim it, naming the oracle being taken.
Contributor guide
Research direction
Start with prompts/help-wanted/correctness-fuzzers.md, then inspect test/fuzz/fuzz_ingest.cpp, test/fuzz/run.sh, test/g1configcheck.sh, and the ripwire_test_csr and ripwire_test_pagerank property tests. Choose one of the five named oracles, write a plan for maintainer approval, observe its gate fail before implementation, and finish with a committed replay corpus and an input-count-bounded gate.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cmake, cpp
- Domain
- build-system, ci-cd, testing-qa
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100