paritytech / paritytech/web3-storage
Replace inline Bucket.members BoundedVec with BucketMembers StorageDoubleMap
@RafalMirowski1 is already working on this.
Since Jul 20, 2026.
- Dominant language
- Rust
- Stars
- 12
- Forks
- 3
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 33
Description
Problem
Bucket.members is a BoundedVec<Member<T>, T::MaxMembers> stored inline in the Bucket struct. With MaxMembers = 100, that's ~3.3 KB (100 × ~33 bytes) decoded on every Buckets::get — including checkpoint extrinsics, which never touch members. Checkpoints are the hot path; this is avoidable PoV/decode cost on a parachain. (We deliberately keep primary_providers inline for efficient checkpoint reads — inline members undermines that same optimization.)
On top of that, every membership check is a linear search over the vec — not the end of the world at a hundred entries, but far from ideal for what is fundamentally a keyed lookup.
On-chain access patterns are almost entirely point lookups:
ensure_admin/ensure_writer_or_admin(auth)- private-bucket challenge membership gate
- last-admin invariant (needs an admin count, not the list)
- full enumeration only off-chain (runtime API → provider membership cache)
Expected cardinality
Cardinality differs sharply by role:
- Admins/Writers: inherently few — many writers is a coordination problem regardless (append ordering, checkpoint signing). If these were the only roles, shrinking
MaxMembersto ~16 and keeping the inline vec would be the simpler fix. - Readers: the read ACL for private buckets. Team-drive and paid-access use cases plausibly need hundreds or more. This is what motivates removing the structural bound rather than tightening it.
Proposal
BucketMembers: StorageDoubleMap<Blake2_128Concat, BucketId, Blake2_128Concat, AccountId, Role>
- Auth checks become one cheap point read; hot checkpoint path drops ~3.3 KB per bucket read/write.
- Removes the structural member cap — membership becomes effectively unbounded, sized by deposits instead.
Alternative considered: hybrid — keep Admin/Writer inline (small bound), move only Readers to a map. Readers are only checked on-chain at the private-challenge gate (a point lookup anyway), so the full map is simpler; hybrid adds a second code path for little gain.
Considerations
- Storage deposit per member — with the bound gone, each
set_membermust reserve a deposit (refunded on removal) to prevent state spam. - Counters in
Bucket— addmember_count: u32andadmin_count: u32, maintained on set/remove. Needed for the last-admin invariant and weight accounting, since iteration is no longer an option. - Bucket deletion —
clear_prefixover an unbounded set can't run in one extrinsic. Use paged removal (limit + cursor over multiple calls) or lazy reaping (tombstoned bucket, anyone can reap entries for the deposit). MemberBucketsreverse index — currentlyAccount -> BoundedVec<BucketId>; nothing on-chain reads it. Either keep it (per-account bound stays fine) or drop it and leave account→buckets enumeration to indexers. Deletion/cleanup must be handled either way.- Off-chain enumeration — runtime API switches to prefix iteration; unbounded is fine off-chain. Provider membership cache should handle large reader sets (pagination in the runtime API response may be needed).
- Migration — storage migration moving existing inline members into the map, plus
Bucketstruct change.
Affected code
pallet/src/lib.rs—Bucketstruct,set_member/remove_memberextrinsicspallet/src/impls/members.rs—ensure_admin,ensure_writer_or_admin,locate_member,set_member_internal,remove_member_internalpallet/src/runtime_api.rs— member enumerationpallet/src/migrations.rsdocs/design/scalable-web3-storage-implementation.md—Bucketdefinition (~line 358)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.