facebookresearch / facebookresearch/SlowFast
Adaptive KV pooling in MViT
Open
- Dominant language
- Python
- Stars
- 7.4k
- Forks
- 1.3k
- PR merge metrics
- No merged PRs in 30d
Description
From what I understand after reading the MViT paper, the concept behind adaptive KV pooling is that the pool stride is reduced (pic from paper attached) when there is a drop in resolution in the input because of applying a non-1 stride to the query in the previous layer. However, https://github.com/facebookresearch/SlowFast/blob/070747fdde88e12bdb0513df352a4e751904c786/slowfast/models/video_model_builder.py#L950 checks if the current query stride is non-1. Is this a bug (i.e., the check should be on the previous query), or am I misunderstanding what's written in the paper?
Contributor guide
Assessment
This issue has not been assessed yet.