google-deepmind / google-deepmind/deepmind-research
implementation of stoch depth in nfnets
Open
- Dominant language
- Jupyter Notebook
- Stars
- 15.2k
- Forks
- 2.9k
- PR merge metrics
- No merged PRs in 30d
Description
The implementation of stoch depth in the code of nfnet seems to be batch-wise dropout, but not block-level dropout as described in paper.
Contributor guide
Assessment
This issue has not been assessed yet.