paritytech / paritytech/web3-storage

Add end-to-end test for the replica flow

Open
#420 0 comments 0 reactions 2 assignees View on GitHub

@danielbui12 is already working on this.

Since Sep 14, 2026.

Dominant language
Rust
Stars
12
Forks
3
Avg merge
2d 2h
Merged PRs (30d)
33

Description

The replica flow from the design (docs/design/scalable-web3-storage-implementation.md, "Adding a replica provider") has no test that runs it end to end. Pallet, replica crate and provider-node each test their layer against mocks. The e2e suite runs one provider and never creates a replica agreement.

Scenario

  1. Start two providers. A is primary, B has replica_sync_price set.
  2. Establish a storage agreement with A, upload data, commit, checkpoint.
  3. Establish a replica agreement with B on the same bucket.
  4. B syncs from A and ends with the on-chain snapshot root.
  5. confirm_replica_sync succeeds: ReplicaSynced event, sync_balance reduced.
  6. challenge_replica against B is answered.

Proposed solution

Add examples/papi/e2e/13-replica-sync.ts and a second start-provider step in the E2E CI job.

Investigate / revisit

Things the test is expected to expose. Check each and fix or file separately:

  • Sync from scratch cannot work: sync_from_primary walks /mmr_peaks and fetches each peak via GET /node. Peaks are MMR node hashes, never stored as content nodes, so the lookup returns 404 for any committed bucket. get_mmr_subtree is still a stub. #392 describes this and was closed without a fix.
  • The node attests only position 0 of the roots array. Positions 1 to 6 (historical roots) stay None, so a checkpoint during a sync makes confirm_replica_sync fail with InvalidSyncRoot instead of matching an older slot.
  • submit_sync_confirmation returns a hard-coded (0, 0) for position_matched and payment instead of reading the ReplicaSynced event, so SyncResult::Success does not reflect the chain.
  • Replica confirmation must match one of the roots from the last ~113 anchor blocks. A busy bucket may never let a replica confirm.
  • A provider cannot be replica and primary of the same bucket, and a replica cannot be promoted to primary.

Related

#392, #340, #342, #65, docs/drafts/RFC_BUCKET_TRANSFER.md.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.