`retire_workers(remove=False)` has no effect?
- Dominant language
- Python
- Stars
- 1.7k
- Forks
- 778
- Avg merge
- 2h 50m
- Merged PRs (30d)
- 3
Description
`remove` is supposed to do this:
https://github.com/dask/distributed/blob/198522bf767d7d8c1927df07951cd34a79109bd0/distributed/scheduler.py#L5971-L5973
In this block, we always call `close_worker`, regardless of `remove`:
https://github.com/dask/distributed/blob/198522bf767d7d8c1927df07951cd34a79109bd0/distributed/scheduler.py#L6093-L6096
But `close_worker` calls `remove_worker` internally:
https://github.com/dask/distributed/blob/198522bf767d7d8c1927df07951cd34a79109bd0/distributed/scheduler.py#L3482-L3494
So if you pass `close_worker=True, remove=False`, the worker will still be immediately removed I think.
For example, this fails:
```diff
diff --git a/distributed/tests/test_active_memory_manager.py b/distributed/tests/test_active_memory_manager.py
index 8fcc3f31..594faa6b 100644
--- a/distributed/tests/test_active_memory_manager.py
+++ b/distributed/tests/test_active_memory_manager.py
@@ -765,7 +765,7 @@ async def test_RetireWorker_no_remove(c, s, a, b):
"""Test RetireWorker behaviour on retire_workers(..., remove=False)"""
x = await c.scatter({"x": "x"}, workers=[a.address])
- await c.retire_workers([a.address], close_workers=False, remove=False)
+ await c.retire_workers([a.address], close_workers=True, remove=False)
# Wait 2 AMM iterations
# retire_workers may return before all keys have been dropped from a
while s.tasks["x"].who_has != {s.workers[b.address]}:
```
Not a very big deal, since this is probably only used internally/in tests. But I'm not sure what the use-case is for `close_worker=True, remove=False`. Maybe we should just get rid of the `remove=` kwarg?
cc @crusaderky
Contributor guide
Assessment
This issue has not been assessed yet.