lightly-ai / lightly-ai/lightly-train
[BUG] <Pre-training weight loading issue>
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.7k
- Forks
- 116
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 6
Description
I attempted to train my dataset using dinov3-eomt semantic segmentation. The code is as follows.
```
import lightly_train
if __name__ == "__main__":
lightly_train.train_semantic_segmentation(
out="/my_experiment",
model="dinov3/vitl16-eomt",
data={
},
"val": {
},
"classes": { # Classes in the dataset
},
# Optional, classes that are in the dataset but should be ignored during
# training.
"ignore_classes": [0],
},
devices=1,
batch_size=16,
)
```
However, when loading the local weight path, the following error occurred. I downloaded the weight file according to the code. How could there be an error? And, when I use the pre-trained weights of dinov2, this problem will not occur.
> Traceback (most recent call last):
> File "/home/user/.conda/envs/seg/lib/python3.11/site-packages/lightly_train/_configs/validate.py", line 30, in pydantic_model_validate
> return model.model_validate(obj)
> ^^^^^^^^^^^^^^^^^^^^^^^^^
> File "/home/user/.conda/envs/seg/lib/python3.11/site-packages/pydantic/main.py", line 716, in model_validate
> return cls.__pydantic_validator__.validate_python(
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> pydantic_core._pydantic_core.ValidationError: 1 validation error for DINOv3EoMTSemanticSegmentationTrainArgs
> load_weights
> Extra inputs are not permitted [type=extra_forbidden, input_value=False, input_type=bool]
> For further information visit https://errors.pydantic.dev/2.12/v/extra_forbidden
> During handling of the above exception, another exception occurred:
> Traceback (most recent call last):
> File "/home/user/cyshi_lx/lightly-train-main/lightly-train-main/train.py", line 4, in
> lightly_train.train_semantic_segmentation(
> File "/home/user/.conda/envs/seg/lib/python3.11/site-packages/lightly_train/_commands/train_task.py", line 443, in train_semantic_segmentation
> return _train_task(config_cls=SemanticSegmentationTrainTaskConfig, **locals())
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> File "/home/user/.conda/envs/seg/lib/python3.11/site-packages/lightly_train/_commands/train_task.py", line 475, in _train_task
> _train_task_from_config(config=config)
> File "/home/user/.conda/envs/seg/lib/python3.11/site-packages/lightly_train/_commands/train_task.py", line 656, in _train_task_from_config
> train_model = train_model_cls(
> ^^^^^^^^^^^^^^^^
> File "/home/user/.conda/envs/seg/lib/python3.11/site-packages/lightly_train/_task_models/dinov3_eomt_semantic_segmentation/train_model.py", line 190, in __init__
> self.model = DINOv3EoMTSemanticSegmentation(
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> File "/home/user/.conda/envs/seg/lib/python3.11/site-packages/lightly_train/_task_models/dinov3_eomt_semantic_segmentation/task_model.py", line 133, in __init__
> backbone = DINOV3_PACKAGE.get_model(
> ^^^^^^^^^^^^^^^^^^^^^^^^^
> File "/home/user/.conda/envs/seg/lib/python3.11/site-packages/lightly_train/_models/dinov3/dinov3_package.py", line 177, in get_model
> model = model_builder(**args)
> ^^^^^^^^^^^^^^^^^^^^^
> File "/home/user/.conda/envs/seg/lib/python3.11/site-packages/lightly_train/_models/dinov3/dinov3_src/hub/backbones.py", line 361, in dinov3_vitl16
> return _make_dinov3_vit(
> ^^^^^^^^^^^^^^^^^
> File "/home/user/.conda/envs/seg/lib/python3.11/site-packages/lightly_train/_models/dinov3/dinov3_src/hub/backbones.py", line 154, in _make_dinov3_vit
> model.load_state_dict(state_dict, strict=True)
> File "/home/user/.conda/envs/seg/lib/python3.11/site-packages/torch/nn/modules/module.py", line 2581, in load_state_dict
> raise RuntimeError(
> RuntimeError: Error(s) in loading state_dict for DinoVisionTransformer:
> Unexpected key(s) in state_dict: "local_cls_norm.weight", "local_cls_norm.bias".
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the failure with the reported dinov3/vitl16-eomt model and local weights, then inspect _models/dinov3/dinov3_src/hub/backbones.py and _models/dinov3/dinov3_package.py around model construction and state-dict loading. Compare the checkpoint keys with the DinoVisionTransformer definition and confirm that compatible pretrained weights load without the reported unexpected keys.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- computer-vision, machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 42/100