google-research / google-research/nested-transformer
This seems like it would be a great option for increasing context window in sequences. Have you tried that yet?
Open
- Dominant language
- Jupyter Notebook
- Stars
- 204
- Forks
- 27
- PR merge metrics
- No merged PRs in 30d
Description
Just suggesting you try a similar architecture with sequences rather than images.
Contributor guide
Assessment
This issue has not been assessed yet.