huggingface / huggingface/open-muse
Maxvit
Open
- Dominant language
- Python
- Stars
- 359
- Forks
- 30
- PR merge metrics
- No merged PRs in 30d
Description
Add in max vit from [here](https://arxiv.org/abs/2204.01697) for high res stage. Code implementation given in [timm](https://github.com/huggingface/pytorch-image-models/blob/main/timm/models/maxxvit.py) and [lucidrian](https://github.com/lucidrains/vit-pytorch/blob/main/vit_pytorch/max_vit.py).
The basic idea is to split self-attention into blocked attention(local) and grid attention(global) so as doing one
As doing global attention is too expensive+doing just local under fits on the data
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.