Helmholtz-AI-Energy / Helmholtz-AI-Energy/propulate
Non-Linear Search Spaces
- Dominant language
- Python
- Stars
- 46
- Forks
- 9
- PR merge metrics
- No merged PRs in 30d
Description
For some use cases, it would be nice to have non-linear search spaces.
For example, for a learning rate search on $[10^{-6}, 10^{-1}]$ we might want to search each order of magnitude equally. However, with a linear search space, propulate searches the larger magnitudes to a much higher degree than the smaller ones as they make up a much larger fraction of the linear search space.
As a workaround, one can convert to a logarithmic scale user-side, i.e., changing the search space to $[-6, -1]$ and converting to the actual learning rate within the `loss_fn`. However, it would be nice quality-of-life improvement if propulate could handle this internally.
Contributor guide
Research direction
Start by reading the search-space API and the loss_fn workaround described in the issue. Determine how a non-linear or logarithmic range should be represented and converted internally, then verify that values across each order of magnitude are sampled comparably while preserving existing linear spaces.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100