should both: attention_mask and global_attention_mask be used for classification?
Open
- Dominant language
- Python
- Stars
- 2.2k
- Forks
- 285
- PR merge metrics
- No merged PRs in 30d
Description
Hi,
Again a conceptual question on text classification.
Since global attention is used on `````` only, I am slightly confused if I should just pass global_attention_mask to the model or both: attention_mask and global_attention_mask. I follow that attention_mask is mainly used to mask the `````` tokens but does it mean ``` n ``` 2 complexity for local attention?
Thanks!
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.