Edge case with wsV2 and high throughput subscriptions
- Dominant language
- Python
- Stars
- 5.5k
- Forks
- 1.7k
- Avg merge
- 3d 10h
- Merged PRs (30d)
- 2
Description
### What was wrong?
- There's an edge case for `WebsocketProvider` (V2) with high throughput subscriptions (e.g. newPendingTransactions) where if there is even a single call for each transaction, there is enough wait time on calls where the subscription queue will get backed up and eventually the `_listen_event` will trigger and not work as it should. It basically bricks the listener task and the only current way to bandaid "fix" it is to increase the cache size - though this isn't sustainable because it will eventually fill up to that higher number as well and brick again.
### How can it be fixed?
Look into the way the `_listen_event` (asyncio.Event) is working and figure out a good way for the waiting to work with this scenario of a full subscription queue that keeps getting more full. Maybe wait until the queue is half full to start listening to messages again? An arbitrary wait time like 1 second? We'll have to play around with this and find a good balance.
Note: There's a good chance that the `asyncio.Queue` will work well enough without this idea of a listen event. Experiment taking out the listen event altogether.
Contributor guide
Assessment
This issue has not been assessed yet.