google-research

google-research/nested-transformer

Nested Hierarchical Transformer https://arxiv.org/pdf/2105.12723.pdf

Stars: 204
Forks: 27
Open beginner issues: 0
Indexed issues: 7

Dominant language: Jupyter Notebook
License: Apache-2.0
Last GitHub push: Mar 3, 2026
Latest indexed: Sep 16, 2026
Contributing guide: Contributing guide
Code of conduct: No code of conduct
Beginner labels: No beginner labels indexed
PR merge metrics: No merged PRs in 30d

Open All

7 open issues indexed

Model Converge Problem Open

google-research/nested-transformer#5 · 8 comments · 0 reactions · 0 assignees · Indexed Sep 16, 2026
Can you make the Cifar training log public? Open

google-research/nested-transformer#6 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 16, 2026
Regarding GradCAT implementation Open

google-research/nested-transformer#7 · 3 comments · 3 reactions · 0 assignees · Indexed Sep 16, 2026
Training hours & Imagenet accuracy Open

google-research/nested-transformer#8 · 3 comments · 0 reactions · 0 assignees · Indexed Sep 16, 2026
This seems like it would be a great option for increasing context window in sequences. Have you tried that yet? Open

google-research/nested-transformer#9 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 16, 2026
PosEmbed is different for each block - shouldn't it be the same!? Open

google-research/nested-transformer#11 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 16, 2026
Is the bulk of the convergence speedup from the ConvPool?! Open

google-research/nested-transformer#12 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 16, 2026

A short digest of beginner-friendly GitHub issues.