google-deepmind / google-deepmind/sonnet
Is there any compressive transformers pre-trained model releasing plan?
Open
- Dominant language
- Python
- Stars
- 10k
- Forks
- 1.3k
- PR merge metrics
- No merged PRs in 30d
Description
Hi, thx for your guys' great works. Is there any compressive transformers pre-trained model releasing plan? Want to see CT as backbone of GPT2 or BERT on long document like Longformer(a new sparse attention transformer variant)
Contributor guide
Assessment
This issue has not been assessed yet.