automl / automl/SMAC3

Parallelize facade.validate()

Open
#1,027 6 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Python
Stars
1.2k
Forks
245
Avg merge
14h 19m
Merged PRs (30d)
5

Description

I wanted to make a feature request/wish but don't know how to label it as such.

It would be great if the validation of incumbent and default configuration could be run on a a dask runner to parallelize them. Currently, when running on a cluster, the validation runs on the login node, while the worker nodes sit idle.

I saw this comment in `smbo.py`, so maybe this is already planned:
```Python
# TODO: Use submit run for faster evaluation
# self._runner.submit_trial(trial_info=trial)
_, cost, _, _ = self._runner.run(config, **kwargs)
```

Contributor guide

Open the contributing guide

Research direction

Start at facade.validate() and read the related flow in smbo.py, especially the TODO around _runner.submit_trial() and the current _runner.run() call. Trace how incumbent and default configuration validation reaches the runner and how the Dask runner is used. Done means those validations execute on worker nodes instead of the login node, with behavior preserved for cluster execution.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.