facebookresearch / facebookresearch/SlowFast
Why is model state dictionary different in pkl file vs pyth from pytorchvideo?
- Dominant language
- Python
- Stars
- 7.4k
- Forks
- 1.3k
- PR merge metrics
- No merged PRs in 30d
Description
I am trying to implement the slowfast model on live feed. I was able to do so using the demo code and feel that it is not straighforward to use in a custom application. While searching, I found [https://pytorchvideo.org/docs/tutorial_torchhub_detection_inference](url) for `slowfast_r50_detection` and that seems a good source. I was playing with the model and checkpoints to use them offline and noted down below observation
I counted states from the output of `list(checkpoint['model_state'].keys())` where
`checkpoint = torch.load('file mentioned in first column of below table')`
Input checkpoint fileCommentOriginal PyTorchVideo weights (pyth file)Working | 662 statesOriginal pkl file (from the screenshot)It shows missing keys error because of nomenclature issue | 1313 statesOriginal pkl Converted (as per #552)Tensor shape mismatch issue | 1313 statesCustom training weightsIt will show missing keys error because of nomenclature issue | 662 statesCustom training weights convertedNeed to change detection head in model as per number of classes | 662 states
I am referring to the nomenclature convention issue mentioned in #552 Issue 1.
The pkl file is the pytorch checkpoint (not the caffe2) from below link in the model zoo.

The pyth checkpoint from pytorchvideo is from this link: [https://dl.fbaipublicfiles.com/pytorchvideo/model_zoo/ava/SLOWFAST_8x8_R50_DETECTION.pyth
](url)
Why is there a lot more weights in the pkl file compared to the pyth file. Even weirder, if I train the model on a custom dataset, it has the same number of states (662) as the original pyth. Could somebody help answer it? Thanks in advance.
Contributor guide
Assessment
This issue has not been assessed yet.