huggingface / huggingface/transformers

🌟 New model addition: FNet

Open
#12,411 5 comments 0 reactions 0 assignees View on GitHub
New model
Dominant language
Python
Stars
166k
Forks
34.6k
Avg merge
3d 8h
Merged PRs (30d)
276

Description

# 🌟 New model addition: FNet

FNet is a highly efficient Transformer-like encoder architecture, wherein the self-attention sublayers have been wholly replaced by standard, unparameterized Fourier Transforms.

I would like to help adding this!

## Open source status

* [x] the model implementation is available: https://github.com/google-research/google-research/tree/master/f_net
* [x] the model weights are available: https://github.com/google-research/google-research/tree/master/f_net
* [x] who are the authors: (@ilyaeck @santiontanon) (Not sure, googled the authors' name + github, sorry if it's incorrect)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.