microsoft / microsoft/FLAML

tune: Warm start using intermediate reports

Open
#814 9 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
4.4k
Forks
565
Avg merge
5d 8m
Merged PRs (30d)
17

Description

Hi,

As per [the docs](https://microsoft.github.io/FLAML/docs/Use-Cases/Tune-User-Defined-Function#warm-start), warm start can be done by passing lists of param configs and, respectively, the associated metrics as `tune.run`'s `points_to_evaluate` and `evalulated_rewards` arguments.

1. Can this mechanism be used for pruned trials? I.e., where is the place of the resource? I'm thinking to pass a list of dicts as returned by the func to evaluate (`{resource_attr: resource, "loss": test_loss}` from the code example in [this](https://microsoft.github.io/FLAML/docs/Use-Cases/Tune-User-Defined-Function#1-an-authentic-scheduler-implemented-in-flaml-schedulerflaml) section of the docs) as `evalulated_rewards`
2. How does one handle intermediate reports? I'm thinking something similar as above, but with multiple identical entries in the `points_to_evaluate` list, but with different `metric/resource` pars in the `evaluated_rewards`.

Now, does this make sense? Even more importantly, does this work :)?

I mean, I imagine this not being the only way to address this. E.g., initially, I was thinking that it makes sense to treat the `resource` just as another parameter in each config. As such, perhaps a small clarification in the docs might be welcome by some.

Thanks

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.