kubeflow / kubeflow/trainer

KEP-3562: [OptimizationJob] Add Normal and LogNormal distribution support to SearchSpace API

Open
#3,795 3 comments 0 reactions 1 assignee Claimed by @sanskar-singh-2403 View on GitHub
area/hpo kind/feature kind/plan-kep
Dominant language
Go
Stars
2.2k
Forks
1.1k
Avg merge
3d 22h
Merged PRs (30d)
39

Description

### What you would like to be added?

Context:
In [KEP-3562](https://github.com/kubeflow/trainer/pull/3565), Phase 1 of `OptimizationJob` supports `Uniform`, `LogUniform`, and `Categorical` search space distributions.

Problem:
Many hyperparameter tuning workloads (such as neural network learning rate warmups or weight initialization factors) sample more effectively from Gaussian/Normal distributions centered around specific mean and standard deviation values.

Future Goals:

Extend `SearchSpace` in `pkg/apis/trainer/v1alpha1/optimizationjob_types.go` to include Normal and LogNormal structs.

Add CEL (XValidation) rules for verifying parameters like mean and stddev.

Propagate continuous probability distributions into the Optuna gRPC suggestion adapter.

Relates to KEP-3562 Master Tracking [Issue](https://github.com/kubeflow/trainer/issues/3562).

### Why is this needed?

Adding `Normal` and `LogNormal` distributions allows practitioners to encode prior domain knowledge into hyperparameter tuning, leading to faster convergence than uniform sampling.

### Love this feature?

Give it a 👍 We prioritize the features with most 👍

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.