paritytech / paritytech/web3-storage
[Provider] Performance SLOs and a regression harness for the provider node
@dr333ws is already working on this.
Since Jul 2, 2026.
- Dominant language
- Rust
- Stars
- 12
- Forks
- 3
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 33
Description
Motivation
"Performant" is currently unmeasured: the provider node has no benchmarks, no performance targets, and no CI signal that a change made uploads, commits, or proof generation slower. Meanwhile we already know the hot paths have structural problems (the MMR is fully rebuilt on every commit and every proof — #100), and the fix work needs numbers to be evaluated against. Without a harness, every performance claim — including "this refactor didn't regress challenge response" — is guesswork, and challenge response latency is stake-at-risk, not cosmetics.
Current state
- No
benches/directory and nocriteriondependency anywhere in the workspace; the only performance-adjacent tooling is FRAME weight benchmarking for extrinsics (on-chain costs, not provider throughput). - No defined targets: nothing states what upload throughput, commit latency, or proof-generation p99 a provider must sustain, so there is nothing to test against (also blocks #42's "provider min requirements" and #48's disk sizing).
- #100/#101 propose one-off measurements to pick a storage architecture/engine — valuable, but one-off; nothing institutionalizes the numbers or catches regressions after the decision.
- #175's
storage-cli stress-test uploadis the natural load-generation vehicle but isn't built yet.
Proposed / potential solution
- Define SLOs first (a short doc, then encode as thresholds): e.g. sustained upload MB/s per stream,
commitlatency vs bucket size, MMR/chunk-proof generation p99 vs leaf count, challenge response end-to-end budget (poll → proof → tx submitted), startup/recovery time vs stored bytes. - Micro-benches:
criterionbenches for the hot paths (mmrappend/proof,storage/diskcommit + proof generation, auth/signing) parameterized by leaf count (1k/10k/100k) so the O(n)-rebuild problem of #100 becomes a visible curve, and its fix a visible win. - Macro harness: build
storage-cli stress-test(#175) into a repeatable scenario runner (N buckets × M uploads × concurrent reads + a challenge in flight) against a real provider + zombienet chain; emit machine-readable results. - CI wiring: a periodic (nightly/weekly, not per-PR) job that runs both layers on a fixed-spec runner, stores results, and alerts on threshold breach or significant delta; per-PR runs opt-in via the
/cmdbot.
TODO
- Triage and prepare a partial design of what to implement (which SLOs, benches and harness pieces come first)
- SLO doc with initial target numbers (inputs for #42 hardware requirements and #48 Paseo sizing)
- criterion micro-benches for mmr/commit/proof paths, parameterized by leaf count
-
storage-cli stress-testscenario runner with machine-readable output (#175) - Nightly perf CI job + regression alerting;
/cmdhook for on-demand PR runs - Baseline report on current main (pre-#100 numbers, so the improvement is documented)
Related
- #100 / #101 — the storage-architecture decisions this harness must measure (before/after)
- #175 —
storage-clistress-test command (load generator) - #214 — telemetry; the same metrics should be exported by production providers
- #42 — provider hardware requirements / cost-benefit needs these numbers
- #113 — 2s blocks change the challenge/checkpoint timing budget
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.