use zmq.PROBE_ROUTER to register connections to schedulers
Open
- Dominant language
- Jupyter Notebook
- Stars
- 2.6k
- Forks
- 1k
- PR merge metrics
- No merged PRs in 30d
Description
There is a race where engine connections to schedulers takes a finite amount of time, and it is possible for clients to be informed of an engine's registration *before* the engine is fully connected to all schedulers.
The current registration design effectively assumes heartbeats will complete after registration is complete, but the faster launch-then-use nature of the cluster Python API reveals the race.
Using the zmq.ROUTER_PROBE should allow us to delay registration notification until all probe messages have been received by the schedulers.
Contributor guide
Assessment
This issue has not been assessed yet.