LocalCluster fails to spawn requested number of workers (capped at 5 workers on Windows)
- Dominant language
- Python
- Stars
- 1.7k
- Forks
- 778
- Avg merge
- 2h 50m
- Merged PRs (30d)
- 3
Description
### Description
When creating a `LocalCluster` with `n_workers` > 5, the cluster consistently only spawns exactly 5 workers, regardless of the requested number. This appears to be a hard limit on Windows systems.
### Environment
- **OS**: Windows
- **Python version**: (output from `python --version`)
- **Dask version**: 2025.10.0
- **Distributed version**: 2025.10.0
- **Installation method**: conda/miniforge
### Minimal Reproducible Example
```python
from dask.distributed import LocalCluster, Client
import time
if __name__ == '__main__':
print('=== TESTING DASK LOCAL CLUSTER WITHOUT PLUGINS ===')
for n_workers in [5, 7, 10, 20, 50]:
print(f'\n--- Testing {n_workers} workers ---')
cluster = LocalCluster(
n_workers=n_workers,
threads_per_worker=1,
processes=True,
memory_limit='500MB',
silence_logs=True
)
client = Client(cluster)
# Wait for workers to connect
time.sleep(5)
# Check how many workers registered
info = client.scheduler_info()
num_connected = len(info['workers'])
print(f'Requested: {n_workers}, Connected: {num_connected}')
client.close()
cluster.close()
time.sleep(2)
print('\n=== TEST COMPLETE ===')
```
### Expected Behavior
The LocalCluster should spawn the number of workers specified by the n_workers parameter. For example:
n_workers=7 should create 7 workers
n_workers=10 should create 10 workers
n_workers=50 should create 50 workers
### Actual Behavior
Regardless of the n_workers parameter, exactly 5 workers are created and connected:
=== TESTING DASK LOCAL CLUSTER WITHOUT PLUGINS ===
--- Testing 5 workers ---
Requested: 5, Connected: 5
--- Testing 7 workers ---
Requested: 7, Connected: 5
--- Testing 10 workers ---
Requested: 10, Connected: 5
--- Testing 20 workers ---
Requested: 20, Connected: 5
--- Testing 50 workers ---
Requested: 50, Connected: 5
=== TEST COMPLETE ===
Contributor guide
Assessment
This issue has not been assessed yet.