paritytech / paritytech/web3-storage
[Tooling] Add a `storage-cli` tooling crate (on-chain + off-chain ops), starting with `stress-test upload`
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 12
- Forks
- 3
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 33
Description
Motivation
We've accumulated a bunch of useful commands and snippets for exercising on-chain
and off-chain storage scenarios (registering providers, opening agreements,
uploading data, etc.), currently scattered across examples and ad-hoc scripts.
Consolidate them into one ergonomic binary so contributors and CI can drive both
layers from a single tool.
Proposal
Add a new workspace crate storage-cli under utils/storage-cli, built on
the existing storage-client SDK.
- New crate
utils/storage-cli, added to[workspace].membersin the rootCargo.toml.
(utils/follows the Polkadot SDK convention of keeping developer tooling under
substrate/utils/.) clap-based CLI (derive API), one subcommand per operation, with shared global flags:--chain-rpc <URL>(defaultws://127.0.0.1:2222)--provider-url <URL>(provider HTTP endpoint; defaulthttp://127.0.0.1:3333)--suri <SURI>/--keyfile <FILE>for the signing account
- Reuse
storage-client(StorageUser, discovery, substrate helpers) instead of
re-implementing chain/HTTP access.
First command: stress-test upload
Simplest case — "upload data to one provider".
Behavior
- Resolve target buckets from chain for the (client, provider) pair: read the
client's buckets (MemberBuckets[client]) and keep those that have an active
StorageAgreements[bucket][provider]entry (i.e. an agreement between this
client and the given provider). - If no matching buckets exist -> exit with a clear error. Do not create any
bucket or agreement. - Upload generated data to each selected bucket via the provider's HTTP endpoint
(StorageUser::upload).
Params
--provider <AccountId>(required) — the provider to target.--max-buckets-to-write <N>(optional) — cap the number of buckets written to;
default = all matching buckets.--size <BYTES>(optional, suggested) — amount of data to generate per upload,
with a sensible default.
Acceptance criteria
-
storage-clicrate compiles as a workspace member;cargo run -p storage-cli -- --help
lists thestress-testsubcommand. -
stress-test upload --provider <id>uploads to all matching buckets;
--max-buckets-to-write Ncaps the count. - Errors clearly when the client has no agreements with the given provider
(no silent success, no bucket/agreement creation). - Reuses
storage-client; no duplicated chain or HTTP logic. -
--helpand a short README document the flow and the required on-chain setup.
Follow-ups (out of scope here)
- More
stress-testmodes: replicated upload, multi-provider fan-out, sustained
load / concurrency. - On-chain helper subcommands (register provider, open agreement, submit checkpoint),
folded in from the existingclient/examples/*.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the workspace members in the root Cargo.toml, the existing storage-client SDK, and relevant client/examples/* commands. Create utils/storage-cli with clap-based entry points, then run cargo run -p storage-cli -- --help and exercise stress-test upload against matching agreements. Done means uploads are capped correctly, missing agreements produce a clear error without creating resources, and the README documents setup and flow.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- cli, tooling
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100