NVIDIA-Merlin / NVIDIA-Merlin/Transformers4Rec
[BUG] Model overfitting when using MaskedLanguage task.
Open
@sararb is already working on this.
Since Nov 18, 2021.
area/tensorflow
bug
P1
- Dominant language
- Python
- Stars
- 1.3k
- Forks
- 165
- Avg merge
- 1m
- Merged PRs (30d)
- 2
Description
Bug description
There Is an information leakage when training TF XLNet model using mlm task. The returned ranking metrics are close to 1.
Steps/Code to reproduce bug
- Run the notebook
03-session-based-XLNet-with-TF.ipynbwithmlmtask instead ofclm
Environment details
- Transformers4Rec version: the latest version of the main branch
- Python version: 3.8.10
- Huggingface Transformers version: 4.12.3
- PyTorch version (GPU?): None
- Tensorflow version (GPU?): 2.6.0
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.