tune: Warm start using intermediate reports
- Dominant language
- Jupyter Notebook
- Stars
- 4.4k
- Forks
- 565
- Avg merge
- 5d 8m
- Merged PRs (30d)
- 17
Description
Hi,
As per [the docs](https://microsoft.github.io/FLAML/docs/Use-Cases/Tune-User-Defined-Function#warm-start), warm start can be done by passing lists of param configs and, respectively, the associated metrics as `tune.run`'s `points_to_evaluate` and `evalulated_rewards` arguments.
1. Can this mechanism be used for pruned trials? I.e., where is the place of the resource? I'm thinking to pass a list of dicts as returned by the func to evaluate (`{resource_attr: resource, "loss": test_loss}` from the code example in [this](https://microsoft.github.io/FLAML/docs/Use-Cases/Tune-User-Defined-Function#1-an-authentic-scheduler-implemented-in-flaml-schedulerflaml) section of the docs) as `evalulated_rewards`
2. How does one handle intermediate reports? I'm thinking something similar as above, but with multiple identical entries in the `points_to_evaluate` list, but with different `metric/resource` pars in the `evaluated_rewards`.
Now, does this make sense? Even more importantly, does this work :)?
I mean, I imagine this not being the only way to address this. E.g., initially, I was thinking that it makes sense to treat the `resource` just as another parameter in each config. As such, perhaps a small clarification in the docs might be welcome by some.
Thanks
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.