argosopentech / argosopentech/argos-train
Pytorch weights_only issue
- Dominant language
- Python
- Stars
- 158
- Forks
- 29
- PR merge metrics
- No merged PRs in 30d
Description
```
Traceback (most recent call last):
File "/home/argosopentech/argos-train/./../OpenNMT-py/tools/average_models.py", line 6, in
main()
File "/home/argosopentech/OpenNMT-py/onmt/bin/average_models.py", line 49, in main
final = average_models(opt.models, opt.fp32)
File "/home/argosopentech/OpenNMT-py/onmt/bin/average_models.py", line 13, in average_models
m = torch.load(model_file, map_location='cpu')
File "/home/argosopentech/env/lib/python3.10/site-packages/torch/serialization.py", line 1529, in load
raise pickle.UnpicklingError(_get_wo_message(str(e))) from None
_pickle.UnpicklingError: Weights only load failed. This file can still be loaded, to do so you have two options, do those steps only if you t
rust the source of the checkpoint.
(1) In PyTorch 2.6, we changed the default value of the `weights_only` argument in `torch.load` from `False` to `True`. Re-running `t
orch.load` with `weights_only` set to `False` will likely succeed, but it can result in arbitrary code execution. Do it only if you got the f
ile from a trusted source.
(2) Alternatively, to load with `weights_only=True` please check the recommended steps in the following error message.
WeightsUnpickler error: Unsupported global: GLOBAL onmt.inputters.text_dataset.TextMultiField was not an allowed global by default. P
lease use `torch.serialization.add_safe_globals([onmt.inputters.text_dataset.TextMultiField])` or the `torch.serialization.safe_globals([onmt
.inputters.text_dataset.TextMultiField])` context manager to allowlist this global if you trust this class/function.
```
Contributor guide
No contributing guide indexed for this repository
Research direction
Start in onmt/bin/average_models.py at the torch.load call shown in the traceback, and reproduce the failure with the reported PyTorch 2.6 behavior and a trusted checkpoint. Determine the compatible loading behavior for model averaging; done means average_models.py can load the checkpoint and complete averaging without this error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100