kubeflow / kubeflow/trainer

Support user‑specified initializers in TrainJob when runtime has none

Open
#2,886 11 comments 3 reactions 1 assignee Claimed by @akshaychitneni View on GitHub
Dominant language
Go
Stars
2.2k
Forks
1.1k
Avg merge
3d 22h
Merged PRs (30d)
39

Description

**What would you like to be added?**
Shall we allow AI Practitioners to set [dataset and model initializers](https://github.com/kubeflow/trainer/blob/master/manifests/base/runtimes/torchtune/llama3_2/llama3_2_1B.yaml#L15-L56) directly on a `TrainJob` even when the selected `TrainingRuntime` or `ClusterTrainingRuntime` has none?

This could be exposed via the Trainer SDK [options](https://github.com/kubeflow/sdk/issues/92) - like `WithInitializer()` and `WithDatasetInitializer()`,`WithModelInitializer()`.

This enables running only a data initializer, only a model initializer, or both, without creating multiple runtimes. But AI Practitioners will be responsible to set Volumes / volumeMounts, image etc.

cc @kubeflow/kubeflow-sdk-team @kubeflow/kubeflow-trainer-team

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.