facebookresearch / facebookresearch/SlowFast

Adaptive KV pooling in MViT

Open
#669 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
7.4k
Forks
1.3k
PR merge metrics
No merged PRs in 30d

Description

From what I understand after reading the MViT paper, the concept behind adaptive KV pooling is that the pool stride is reduced (pic from paper attached) when there is a drop in resolution in the input because of applying a non-1 stride to the query in the previous layer. However, https://github.com/facebookresearch/SlowFast/blob/070747fdde88e12bdb0513df352a4e751904c786/slowfast/models/video_model_builder.py#L950 checks if the current query stride is non-1. Is this a bug (i.e., the check should be on the previous query), or am I misunderstanding what's written in the paper?

Screenshot 2023-08-24 at 2 08 00 PM

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.