HuggingFaceModelInitializer and S3ModelInitializer share a mutable default ignore_patterns list across instances
- Dominant language
- Python
- Stars
- 148
- Forks
- 262
- Avg merge
- 1d 2h
- Merged PRs (30d)
- 1
Description
### What happened?
Description
While reviewing kubeflow/trainer/types/types.py, I noticed that both HuggingFaceModelInitializer and S3ModelInitializer define ignore_patterns as:
ignore_patterns: list[str] | None = field(
default_factory=lambda: constants.INITIALIZER_DEFAULT_IGNORE_PATTERNS
)
At first glance this looks correct because default_factory is used. However, the factory returns the module-level mutable list directly instead of creating a new list for each instance.
INITIALIZER_DEFAULT_IGNORE_PATTERNS is defined as:
INITIALIZER_DEFAULT_IGNORE_PATTERNS = [
"*.msgpack",
"*.h5",
"*.bin",
"*.pt",
"*.pth",
]
Since the same list object is returned every time, all initializer instances share the same underlying ignore_patterns list.
Verified locally
from kubeflow.trainer.types.types import HuggingFaceModelInitializer
a = HuggingFaceModelInitializer(storage_uri="hf://user/model1")
b = HuggingFaceModelInitializer(storage_uri="hf://user/model2")
print(a.ignore_patterns is b.ignore_patterns)
Output:
True
Both instances reference the exact same list object.
To verify the impact:
print("Before:")
print("A:", a.ignore_patterns)
print("B:", b.ignore_patterns)
a.ignore_patterns.append("*.custom")
print("After:")
print("A:", a.ignore_patterns)
print("B:", b.ignore_patterns)
Output:
Before:
A: ['*.msgpack', '*.h5', '*.bin', '*.pt', '*.pth']
B: ['*.msgpack', '*.h5', '*.bin', '*.pt', '*.pth']
After:
A: ['*.msgpack', '*.h5', '*.bin', '*.pt', '*.pth', '*.custom']
B: ['*.msgpack', '*.h5', '*.bin', '*.pt', '*.pth', '*.custom']
Even though only a.ignore_patterns was modified, the change is immediately visible in b.ignore_patterns because both instances share the same mutable list.
Existing pattern in the codebase
The same file already uses the correct pattern for CustomTrainer.pip_index_urls:
pip_index_urls: list[str] = field(
default_factory=lambda: list(constants.DEFAULT_PIP_INDEX_URLS)
)
Creating a new list for every instance avoids sharing mutable state.
Expected behavior
Each HuggingFaceModelInitializer and S3ModelInitializer instance should receive its own independent copy of the default ignore_patterns list.
Suggested fix
ignore_patterns: list[str] | None = field(
default_factory=lambda: list(constants.INITIALIZER_DEFAULT_IGNORE_PATTERNS)
)
This matches the pattern already used elsewhere in the same file.
Environment
Repository: kubeflow/sdk
File: kubeflow/trainer/types/types.py
Affected classes:
HuggingFaceModelInitializer
S3ModelInitializer
I'd be happy to submit a PR if this behavior is considered a bug.
### What did you expect to happen?
Each `HuggingFaceModelInitializer` and `S3ModelInitializer` instance should receive its own independent copy of the default `ignore_patterns` list.
Mutating the `ignore_patterns` of one instance should not affect any other initializer instance.
For example, after creating two independent initializer objects, appending a new ignore pattern to one instance should leave the other instance unchanged.
### Environment
Kubernetes version:
```bash
$ kubectl version
```
Kubeflow Trainer version:
```bash
$ kubectl get pods -n kubeflow -l app.kubernetes.io/name=trainer -o jsonpath="{.items[*].spec.containers[*].image}"
```
Kubeflow Python SDK version:
```bash
$ pip show kubeflow
```
### Impacted by this bug?
Give it a 👍 We prioritize the issues with most 👍
Contributor guide
Research direction
Start in kubeflow/trainer/types/types.py at HuggingFaceModelInitializer and S3ModelInitializer, then compare their ignore_patterns definitions with CustomTrainer.pip_index_urls. Verify that separately created initializers no longer share mutable default state, and confirm that changing one instance leaves the other unchanged.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- huggingface, python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 78/100