paritytech / paritytech/web3-storage

[Provider] Bucket quota (max_bytes) is never enforced — every bucket is initialised as unlimited

Open
#382 1 comment 0 reactions 1 assignee View on GitHub

@ilchu is already working on this.

Since Sep 8, 2026.

Dominant language
Rust
Stars
12
Forks
3
Avg merge
2d 2h
Merged PRs (30d)
33

Description

Problem

Storage agreements sell a byte quota. The client prepays price_per_byte × max_bytes × duration, the provider's stake must cover MinStakePerByte × committed_bytes, and the pallet refuses agreements exceeding the provider's max_capacity.

The provider node ignores all of it. Every bucket is created with an unlimited quota, so a client who paid for 1 GB can upload 100 GB. The provider serves it, is slashable for it, and is staked for 1 GB.

The enforcement code exists and is correct — it just never gets a real limit to enforce.

What the design says

The design specifies provider-side quota enforcement, down to the wire error:

  • "Providers accept all uploads as long as the bucket has quota" (stated twice)
  • "Maximum bytes (quota) — provider accepts uploads up to this"
  • "Users who create conflicts without checkpointing waste their quota"
  • Documented response: 507 Insufficient Storage with { "error": "quota_exceeded", "used": …, "max": … }

See docs/design/scalable-web3-storage-implementation.md. That response is fully implemented in provider-node/src/error.rs — correct status, correct body — and unreachable. So this is drift from the design, not a missing design decision.

Starting points

  • The quota check lives in store_node in the RocksDB backend (crates/providers/storage/src/backend/rocksdb.rs). It is correct and never triggered.
  • Every production caller of init_bucket passes u64::MAX as the quota: the /node upload handler, the S3 put-object handler, the FS put-file handler, and replica sync.
  • Nothing plumbs the on-chain StorageAgreement.max_bytes into storage — the node reads that field only to build the terms it signs.

Task

Analyse and propose the fix. Worth working out before coding:

  • init_bucket is create-only and StorageBackend has no quota setter, so swapping the constant won't cover buckets already on disk, nor top_up_agreement raising the quota mid-term. Where does the quota come from, and when is it refreshed?
  • What happens when the chain is unreachable at startup — fail closed, or serve on the last known quota?
  • Should the S3/FS gateway paths enforce it, or only the L0 /node path?
  • Deduplication: the check adds the incoming length, but used_bytes only increments for new hashes, and the nodes column family is keyed by content hash with no bucket namespace — so a chunk another bucket already holds is stored uncharged. Decide what used_bytes should count (touches the refcount/GC question in #100).
  • Replica agreements carry their own max_bytes — how do replica buckets get theirs?

Output: a short proposal on the above, then the fix plus tests:

  1. An upload that would exceed the bucket's max_bytes is rejected with 507 and quota_exceeded, and nothing is written.
  2. After top_up_agreement, the same upload succeeds — the node picked up the raised quota.
  3. The quota survives a node restart (not silently reset to unlimited).

Related

Precedes #379 — that issue proposes a third provider-declared limit (max_buckets, min_price) while the byte limit already declared, priced and staked against isn't enforced. The dedup item overlaps #100 and the PR #253 review.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.