Instantiate a JobQueueCluster with an already started Scheduler
- Dominant language
- Python
- Stars
- 256
- Forks
- 150
- PR merge metrics
- No merged PRs in 30d
Description
While looking in several issues here, I feel that adding the ability to instanciate a JobQueueCluster with an already started Scheduler could help some use cases, and be a first step towards starting the Scheduler remotly on a job (#186).
The idea would be to add the possibility to do:
```python
cluster = FooCluster(scheduler=my_scheduler)
#or
cluster = FooCluster(scheduler_info=scheduler_file)
```
Then the JobQueueCluster won't instantiate a Scheduler, but just connect to an existing one.
I think this woul provide solutions to some issues:
* #407, we could reuse a LocalCluster Scheduler :
```python
loc_cluster = LocalCluster(...)
job_cluster = FooCluster(scheduler=loc_cluster.scheduler)
```
* #378, #381 :
```python
lowmem_cluster = FooCluster(cores=2,memory='4GB',resources=...)
highmem_cluster = FooCluster(scheduler=lowmeme_cluster.scheduler,cores=2,memory='16GB',resources=...)
```
Any thought?
Contributor guide
Assessment
This issue has not been assessed yet.