NVIDIA-Merlin / NVIDIA-Merlin/Transformers4Rec
[BUG] `load_model_trainer_states_from_checkpoint()` function is broken
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.3k
- Forks
- 165
- Avg merge
- 1m
- Merged PRs (30d)
- 2
Description
Bug description
The transformers4rec.torch.trainer.load_model_trainer_states_from_checkpoint() function can only load t4rec_model_class.pkl, all the other loads are broken. The utils.serialization.load() function is called 4 times in the function, but only one of them successfully loads, the other throw errors. I believe this is because load() is expecting an _io.BufferedReader object (resulting from open()) instead of a path.
Correctly implemented:
https://github.com/NVIDIA-Merlin/Transformers4Rec/blob/5d59d14562cff667ce624f4a2a619a9803b7c4af/transformers4rec/torch/trainer.py#L744-L746
Incorrectly implemented:
https://github.com/NVIDIA-Merlin/Transformers4Rec/blob/5d59d14562cff667ce624f4a2a619a9803b7c4af/transformers4rec/torch/trainer.py#L753
https://github.com/NVIDIA-Merlin/Transformers4Rec/blob/5d59d14562cff667ce624f4a2a619a9803b7c4af/transformers4rec/torch/trainer.py#L757
https://github.com/NVIDIA-Merlin/Transformers4Rec/blob/5d59d14562cff667ce624f4a2a619a9803b7c4af/transformers4rec/torch/trainer.py#L766
Steps/Code to reproduce bug
Run the following code with a Model checkpoint at /checkpoint:
from transformers4rec import torch as tr
trainer = tr.trainer.Trainer(model=tr.model.base.Model(), args=None)
trainer.load_model_trainer_states_from_checkpoint('/checkpoint')
Expected behavior
The function call shouldn't be erroring.
Environment details
- Transformers4Rec version: 23.12.0
- Platform: Linux
- Python version: 3.10.12
- Huggingface Transformers version: 4.27.1
- PyTorch version (GPU?): 2.1.2 (no)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in transformers4rec/torch/trainer.py at load_model_trainer_states_from_checkpoint(), especially the working load call around lines 744-746 and the other calls around lines 753, 757, and 766. Reproduce with the provided checkpoint-loading example, then verify that all trainer states load without errors rather than only t4rec_model_class.pkl.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 74/100