paritytech / paritytech/web3-storage
Repo structure: split provider-node, break the storage-client dependency, tidy crate layout
@danielbui12 is already working on this.
Since Aug 24, 2026.
- Dominant language
- Rust
- Stars
- 12
- Forks
- 3
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 33
Description
Why
provider-nodedepends onstorage-client— an inverted edge (server pulling in the client SDK).provider-nodeis one ~8.4k-LOC crate mixing storage, HTTP, and on-chain coordinators.- Crate layout is inconsistent (
runtime/vsruntimes/, dir names != crate names, flat vs nested).
Also sets up a clean home for utils/storage-cli (#175).
1. Break provider-node -> storage-client
What it actually pulls from storage-client is two unrelated things:
- Pure wire types (no I/O): negotiate
agreement::{NegotiateRequest, AgreementTermsOf, SignedTerms}+sign_terms+discovery::ProviderInfo, used server-side in the/negotiatehandler. - A second chain client:
chain_state_coordinator.rsusesstorage_client::ProviderClientto talk to the parachain over WS —get_provider_info,fetch_replay_hsn,fetch_request_timeout, plus a finalized-block subscription parsed withStorageProviderEventParser+substrate::PALLET_NAME.
⚠️ The provider already has its own chain client. subxt_client.rs (SubxtChainClient, ~917 LOC) talks to the same parachain directly via subxt. So today there are two parallel chain-access paths in one node — the node's own SubxtChainClient and the reads/subscription routed through storage_client::ProviderClient. Step 1 isn't just deleting a types dependency; it collapses those two clients into one.
- Extract
storage-negotiationprimitive crate (wire types +MultiSignatureserde +sign_terms);AgreementTermsetc. already live inprimitives. - Move the chain reads + event subscription (
get_provider_info,fetch_replay_hsn,fetch_request_timeout, finalized-block stream) onto the node's ownSubxtChainClient(post-#159), retiring theProviderClientpath. - Drop
storage-clientfromprovider-node.
2. Split provider-node -> provider/{storage,rpc,node}
| Crate | Modules | subxt? |
|---|---|---|
provider/storage |
storage/*, fs_index, s3_index, mmr |
no |
provider/rpc (HTTP API) |
api, fs_api, s3_api, auth, negotiate, error, types |
no |
provider/node (bin) |
main, command, cli, lib, subxt_client, coordinators |
yes |
Direction: node -> rpc -> storage -> primitives; only node touches the chain. rpc gets chain data via traits it defines, with subxt-backed impls supplied by node (extends #145).
3. Crate layout
Changed (Rust crates that move / get created)
pallets/ <- consolidates scattered pallets
storage-provider/ was pallet/ (pallet-storage-provider)
drive-registry/ was storage-interfaces/file-system/pallet-registry
s3-registry/ was storage-interfaces/s3/pallet-s3-registry
primitives/
storage/ was primitives/ (storage-primitives)
file-system/ was storage-interfaces/file-system/primitives
s3/ was storage-interfaces/s3/primitives
negotiation/ NEW — wire types, MultiSignature serde, sign_terms (step 1)
provider/ <- was the single provider-node/ crate
storage/ storage/*, fs_index, s3_index, mmr
rpc/ api, fs_api, s3_api, auth(core), negotiate, error, types
node/ main, command, cli, lib, subxt_client, coordinators (the bin)
runtimes/
parachain/ was runtimes/web3-storage-local (storage-parachain-runtime)
paseo/ was runtimes/web3-storage-paseo
clients/ <- was client/ + storage-interfaces/*/client
storage/ was client/ (storage-client)
file-system/ was storage-interfaces/file-system/client
s3/ was storage-interfaces/s3/client
utils/
storage-cli/ NEW (#175)
Net effect: pallet/, primitives/, client/, provider-node/, and the whole
storage-interfaces/ tree are dissolved into the role-based groups above;
runtimes/ subdirs are renamed. Align dir names to crate names while moving
(today every dir mismatches its crate, e.g. client/ -> storage-client).
Unchanged (stay where they are)
precompiles/ drive-registry, s3-registry, storage-provider
packages/ JS/TS SDK: core, layer0, layer1, papi, sdk (the old `sdk/ ???` slot)
user-interfaces/ JS/TS UIs: landing, drive-ui, provider, s3-ui, console-ui, shared
zombienet/ zombienet.toml + local network config
chain-specs/ docs/ examples/ templates/ scripts/ .bin/ .config/ .github/
Their contents don't move, but the mechanical reorg PR must still fix path
references inside scripts/*.sh, zombienet.toml, the justfile, CI
workflows under .github/, and docs that cite moved crate paths.
Note:
runtime/vsruntimes/(in "Why") is already resolved — onlyruntimes/exists today.
Plan (each its own PR; reorg last)
- Break the coupling (step 1)
- Split provider-node —
storagefirst, thenrpc, leavingnodeas the bin - Directory reorg — one mechanical
git mvPR fixing Cargo paths + CI + scripts together
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.