huggingface / huggingface/transformers
🌟 New model addition: FNet
Open
New model
- Dominant language
- Python
- Stars
- 166k
- Forks
- 34.6k
- Avg merge
- 3d 8h
- Merged PRs (30d)
- 276
Description
# 🌟 New model addition: FNet
FNet is a highly efficient Transformer-like encoder architecture, wherein the self-attention sublayers have been wholly replaced by standard, unparameterized Fourier Transforms.
I would like to help adding this!
## Open source status
* [x] the model implementation is available: https://github.com/google-research/google-research/tree/master/f_net
* [x] the model weights are available: https://github.com/google-research/google-research/tree/master/f_net
* [x] who are the authors: (@ilyaeck @santiontanon) (Not sure, googled the authors' name + github, sorry if it's incorrect)
Contributor guide
Assessment
This issue has not been assessed yet.