kubeflow / kubeflow/trainer

Initializer environment loading overrides dataclass defaults with None

Open
#3,993 2 comments 0 reactions 1 assignee Claimed by @DevaanshPathak View on GitHub
kind/bug
Dominant language
Go
Stars
2.2k
Forks
1.1k
Avg merge
3d 22h
Merged PRs (30d)
39

Description

### What happened?

`get_config_from_env()` adds every dataclass field to its returned dictionary, using `None` when the corresponding environment variable is absent. The initializer then expands that dictionary into its configuration dataclass, so an explicit `None` overrides any declared dataclass default.

This affects multiple initializer configurations. For example, with only the required environment variables set:

```text
HuggingFaceModelInitializer.ignore_patterns: None
expected: [*.msgpack, *.h5, *.bin, *.pt, *.pth]

CacheDatasetInitializer.cluster_size: None
expected: 3

CacheDatasetInitializer.head_cpu: None
expected: 1

CacheDatasetInitializer.readiness_period_seconds: None
expected: 10
```

The same behavior applies to the other cache resource and readiness defaults. Existing provider tests generally mock `get_config_from_env()`, so they do not exercise the real environment-to-dataclass path.

### What did you expect to happen?

When an optional environment variable is absent, the corresponding dataclass default should remain in effect. Explicitly provided environment variables should continue to override that default.

### Relevant files

- `pkg/initializers/utils/utils.py`
- `pkg/initializers/types/types.py`
- `pkg/initializers/utils/utils_test.py`

### Proposed fix

Omit absent environment variables from the dictionary returned by `get_config_from_env()` so dataclass construction applies declared defaults naturally. Add regression coverage for model ignore patterns and cache operational defaults through the real environment-loading path.

I would like to work on a focused fix after maintainer confirmation. No PR has been opened yet.

AI assistance disclosure: OpenAI Codex assisted with repository inspection, reproduction, and drafting; I verified the behavior against current master.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.