graykode / graykode/xlnet-Pytorch
Confusion about the relative position embedding with attn_type='bi' but bsz=1
Open
help wanted
question
- Dominant language
- Jupyter Notebook
- Stars
- 581
- Forks
- 103
- PR merge metrics
- No merged PRs in 30d
Description
The default setting is to use the bidirectional data, attn_type='bi', but bsz=1.
But in this function,
https://github.com/graykode/xlnet-Pytorch/blob/cb793a1c75bdc59e3360f04ec641af726719811f/xlnet.py#L371
It shows the bidirectional data only works when bsz%2 ==0. However in default, bsz = 1.
I am confused, if bsz=1, the setting for the beg, and end in the following code, is it right?
https://github.com/graykode/xlnet-Pytorch/blob/cb793a1c75bdc59e3360f04ec641af726719811f/xlnet.py#L380-L387
Could anyone help me with this confusion?
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.