ApeWorX / ApeWorX/web3.py

Edge case with wsV2 and high throughput subscriptions

Open
#3,221 0 comments 0 reactions 0 assignees View on GitHub
priority: p2 normal
Dominant language
Python
Stars
5.5k
Forks
1.7k
Avg merge
3d 10h
Merged PRs (30d)
2

Description

### What was wrong?

- There's an edge case for `WebsocketProvider` (V2) with high throughput subscriptions (e.g. newPendingTransactions) where if there is even a single call for each transaction, there is enough wait time on calls where the subscription queue will get backed up and eventually the `_listen_event` will trigger and not work as it should. It basically bricks the listener task and the only current way to bandaid "fix" it is to increase the cache size - though this isn't sustainable because it will eventually fill up to that higher number as well and brick again.

### How can it be fixed?

Look into the way the `_listen_event` (asyncio.Event) is working and figure out a good way for the waiting to work with this scenario of a full subscription queue that keeps getting more full. Maybe wait until the queue is half full to start listening to messages again? An arbitrary wait time like 1 second? We'll have to play around with this and find a good balance.

Note: There's a good chance that the `asyncio.Queue` will work well enough without this idea of a listen event. Experiment taking out the listen event altogether.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.