`FollowStreamDriverSubscription`'s will lead to events being collected in unbounded VecDeque until polled.
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 489
- Forks
- 293
- Avg merge
- 18h 35m
- Merged PRs (30d)
- 3
Description
This is an issue raised by the auditors.
Simply put, if a user is using the UnstableBackend, then there are Backend calls which create FollowStreamDriverSubscription's. These subscriptions contain a queue of all of the un-consumed events received from the chain. When the FollowStreamDriver is polled (which would often handle in the background), it will continue to receive events from the backend and add them to the queues for any active subscriptions. So, if these subscriptions aren't polled, they will store an ever-growing list of events waiting to be consumed through polling.
The user is expected to poll FollowStreamDriver (actually UnstableBackendDriver, which is the thing they get back when creating an UnstableBackend and contains it) more slowly if they are struggling to keep up, which would enforce backpressure and slow down the rate at which events are obtained from the chain.
To help bound memory usage a little better, we could also consider adding a configuration option to UnstableBackendBuilder like fn max_event_buffer_per_subscription(self, size: usize) -> Self to bound the number of events that can be queued up on any given subscription before it's shut down and cleaned up. We could also consider setting an arbitrary default, like 1024 events, to give breathing room but prevent unlimited growth when nothing is being polled except theUnstableBackendDriver
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing FollowStreamDriverSubscription and its VecDeque through UnstableBackendDriver, then inspect how UnstableBackendBuilder configures subscriptions. Determine the intended behavior when an unpolled subscription reaches a buffer limit, and consider how the subscription is shut down and cleaned up. Done means memory growth is bounded and the behavior is covered by verification for slow or absent polling.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- backend, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100