Helmholtz-AI-Energy / Helmholtz-AI-Energy/propulate

Non-Linear Search Spaces

Open
#101 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
46
Forks
9
PR merge metrics
No merged PRs in 30d

Description

For some use cases, it would be nice to have non-linear search spaces.
For example, for a learning rate search on $[10^{-6}, 10^{-1}]$ we might want to search each order of magnitude equally. However, with a linear search space, propulate searches the larger magnitudes to a much higher degree than the smaller ones as they make up a much larger fraction of the linear search space.
As a workaround, one can convert to a logarithmic scale user-side, i.e., changing the search space to $[-6, -1]$ and converting to the actual learning rate within the `loss_fn`. However, it would be nice quality-of-life improvement if propulate could handle this internally.

Contributor guide

Open the contributing guide

Research direction

Start by reading the search-space API and the loss_fn workaround described in the issue. Determine how a non-linear or logarithmic range should be represented and converted internally, then verify that values across each order of magnitude are sampled comparably while preserving existing linear spaces.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.