aws / aws/sagemaker-python-sdk
Support Conditional Hyperparameter in AMT
- Dominant language
- Python
- Stars
- 2.3k
- Forks
- 1.3k
- Avg merge
- 1d 22h
- Merged PRs (30d)
- 35
Description
**Describe the feature you'd like**
Hyperparameter tuning frameworks like [optuna](https://optuna.org/) and google's [vertex AI](https://cloud.google.com/vertex-ai/docs/training/hyperparameter-tuning-overview#conditional_hyperparameters) all support some form of conditional hyperparameter. Could you consider supporting this feature in some capacity?
**How would this feature be used? Please describe.**
For instance, in training a cnn, we can define hyperparameters conditional on their parent hyperparameter matching some condition:
```
search_space = {}
if optimizer == 'adam':
search_space['adam_beta_1'] = range(...)
search_space['adam_beta_2'] = range(...)
elif optimizer == 'sgd':
...
```
**Describe alternatives you've considered**
Right now there are a few ways to work around this:
* Using optuna with sagemaker
* Enumerate all possible hyperparameters and handle appropriately in the training entry point (i.e., ignore the ones not needed)
It would be very helpful if this feature is supported by default. Thanks.
Contributor guide
Research direction
No files, tests, or entry points are named. Start by reviewing the existing AMT hyperparameter-tuning flow and the linked Optuna and Vertex AI conditional-hyperparameter documentation, then define how the CNN example's conditional search space should be represented and validated.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, python
- Domain
- cloud, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100