graykode / graykode/xlnet-Pytorch
Parameter initialized with torch.randn may be not a good choice
Open
- Dominant language
- Jupyter Notebook
- Stars
- 581
- Forks
- 103
- PR merge metrics
- No merged PRs in 30d
Description
It seems that the parameter initialized with **randn** (https://github.com/graykode/xlnet-Pytorch/blob/cb793a1c75bdc59e3360f04ec641af726719811f/xlnet.py#L119) will **lead to low-performance**, and I tried **xavier_norm** and **kaiming_uniform**, both reach a much higher AUC and F1 score in my task.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.