paritytech / paritytech/subxt

`FollowStreamDriverSubscription`'s will lead to events being collected in unbounded VecDeque until polled.

Open
#1,443 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

good first issue
Dominant language
Rust
Stars
489
Forks
293
Avg merge
18h 35m
Merged PRs (30d)
3

Description

This is an issue raised by the auditors.

Simply put, if a user is using the UnstableBackend, then there are Backend calls which create FollowStreamDriverSubscription's. These subscriptions contain a queue of all of the un-consumed events received from the chain. When the FollowStreamDriver is polled (which would often handle in the background), it will continue to receive events from the backend and add them to the queues for any active subscriptions. So, if these subscriptions aren't polled, they will store an ever-growing list of events waiting to be consumed through polling.

The user is expected to poll FollowStreamDriver (actually UnstableBackendDriver, which is the thing they get back when creating an UnstableBackend and contains it) more slowly if they are struggling to keep up, which would enforce backpressure and slow down the rate at which events are obtained from the chain.

To help bound memory usage a little better, we could also consider adding a configuration option to UnstableBackendBuilder like fn max_event_buffer_per_subscription(self, size: usize) -> Self to bound the number of events that can be queued up on any given subscription before it's shut down and cleaned up. We could also consider setting an arbitrary default, like 1024 events, to give breathing room but prevent unlimited growth when nothing is being polled except theUnstableBackendDriver

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing FollowStreamDriverSubscription and its VecDeque through UnstableBackendDriver, then inspect how UnstableBackendBuilder configures subscriptions. Determine the intended behavior when an unpolled subscription reaches a buffer limit, and consider how the subscription is shut down and cleaned up. Done means memory growth is bounded and the behavior is covered by verification for slow or absent polling.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
backend, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.