dask / dask/distributed

Registering a nanny plugin should fail if workers are not using nannies

Open
#5,231 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.7k
Forks
778
Avg merge
2h 50m
Merged PRs (30d)
3

Description

`Client.register_worker_plugin(..., nanny=True)` seems to succeed if even if no workers are using Nannies, but doesn't actually do anything.

This is misleading to users using new functionality like `UploadDirectory` on clusters that aren't using nannies.

**What you expected to happen**:

`register_worker_plugin` should raise an error (or at least warning) saying that some/all/XXX% of workers are not using nannies, so the plugin will have no effect on those workers.

**Minimal Complete Verifiable Example**:

```python
import os
import distributed

if __name__ == "__main__":
c = distributed.Client(n_workers=1, worker_class=distributed.Worker)
c.register_worker_plugin(distributed.Environ({"foo": "bar"}), nanny=True)

print(c.run(lambda: os.environ.get("foo", None)))
# {'inproc://192.168.0.21/41810/3': None}
```
If you switch to `worker_class=distributed.Nanny`, you'll see "bar" instead.

**Environment**:

- Dask version: 2021.8.0
- Python version: 3.9.1
- Operating System: macOS
- Install method (conda, pip, source): pip

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.