KEP-3562: [OptimizationJob] Implement and refactor Optuna suggestion service backend
- Dominant language
- Go
- Stars
- 2.2k
- Forks
- 1.1k
- Avg merge
- 3d 22h
- Merged PRs (30d)
- 39
Description
### What you would like to be added?
Context:
In [KEP-3562](https://github.com/kubeflow/trainer/pull/3565), Optuna is selected as the primary backend engine for Phase 1 hyperparameter search algorithms (e.g., Random, Grid).
Problem:
The Optuna integration needs to run as an ephemeral sidecar/container service managed directly by the OptimizationJob controller without relying on external databases or stateful storage.
Future Goals:
- Refactor the Optuna wrapper in `kubeflow/trainer` to act as an isolated gRPC service container.
- Ensure in-memory trial tracking within the container during the job's lifecycle.
Relates to KEP-3562 Master Tracking [Issue](https://github.com/kubeflow/trainer/issues/3562).
### Why is this needed?
A clean, self-contained Optuna service ensures low latency during suggestion requests and guarantees clean teardown when the `OptimizationJob` reaches a terminal state (Complete/Failed).
### Love this feature?
Give it a 👍 We prioritize the features with most 👍
Contributor guide
Assessment
This issue has not been assessed yet.