facebookresearch / facebookresearch/SlowFast

How to get the output of features from MViT?

Open
#755 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
7.4k
Forks
1.3k
PR merge metrics
No merged PRs in 30d

Description

Could anyone give me an example.

Here is the code and yaml config file:

```yaml
MVIT:
ZERO_DECAY_POS_CLS: False
NUM_HEADS: 1
EMBED_DIM: 96
PATCH_KERNEL: (3, 7, 7)
PATCH_STRIDE: (2, 4, 4)
PATCH_PADDING: (1, 3, 3)
MLP_RATIO: 4.0
QKV_BIAS: True
DROPOUT_RATE: 0.0
DROPPATH_RATE: 0.2
NORM: "layernorm"
MODE: "conv"
DEPTH: 16
POOL_Q_STRIDE: [[1, 1, 2, 2], [3, 1, 2, 2], [14, 1, 2, 2]]
DIM_MUL: [[1, 2.0], [3, 2.0], [14, 2.0]]
HEAD_MUL: [[1, 2.0], [3, 2.0], [14, 2.0]]
POOL_KV_STRIDE_ADAPTIVE: [1, 8, 8]
POOL_KVQ_KERNEL: [3, 3, 3]
SEP_POS_EMBED: True
CLS_EMBED_ON: True
FREEZE_PATCH: False
```

```python
cfg = get_cfg()
cfg.merge_from_file("/path/to/yaml/file")

fake_imgs = torch.randint(0, 255, (24, 3, 16, 224, 224)).float().cuda()
model = build_model(cfg)
model(fake_imgs)
```

and here is the error message:

```bash
Traceback (most recent call last):
File "/opt/data/private/code/tapis-based-surgicalc2c/test_tapis.py", line 32, in
model(fake_imgs)
File "/opt/data/private/virtualenv/torch_grasp/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "/opt/data/private/code/tapis-based-surgicalc2c/tapis/models/video_model_builder.py", line 856, in forward
x = torch.cat((cls_tokens, x), dim=1)
RuntimeError: Sizes of tensors must match except in dimension 1. Expected size 96 but got size 8 for tensor number 1 in the list.
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.