Include `BatchedSend` state in cluster dumps
- Dominant language
- Python
- Stars
- 1.7k
- Forks
- 778
- Avg merge
- 2h 50m
- Merged PRs (30d)
- 3
Description
`BatchedSend` is known to be brittle, and certain error cases can cause it to drop messages or even shut down entirely, causing deadlocks:
* https://github.com/dask/distributed/issues/5481
* https://github.com/dask/distributed/pull/5457
I don't think we've seen these problems actually come up in the wild since https://github.com/dask/distributed/pull/5525.
But when looking over cluster dumps, it would be nice to be able to rule this out as an issue. So I think we should include some minimal information about the state of BatchedSend, such as:
* `len(buffer)`
* `please_stop`
* `waker.is_set()`
* `next_deadline`
Obviously it would also be nice to include the status of the coroutine, but since we don't have a handle to it, that's lost to the ether of Tornado.
Contributor guide
Assessment
This issue has not been assessed yet.