google-deepmind / google-deepmind/kinetics-i3d

flow_imagenet checkpoint

Open
#102 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.8k
Forks
467
PR merge metrics
No merged PRs in 30d

Description

The readme states:

> The default model has been pre-trained on ImageNet and then Kinetics

As far as I understand, "pre-trained on ImageNet" means 3D convolutional NN which weights are recieved by bootsrapping values from 2D convolutional NN trained on ImageNet. So `data/checkpoints/rgb_imagenet` checkpoint is RGB network which is initialized with bootstrapped weights from 2D NN (which was trained on RGB images from ImageNet) and then trained on Kinetics dataset. Please, correct me if there are any mistakes.

If the description above is correct then I'm not sure I understand how `data/checkpoints/flow_imagenet` weights were achieved. I think it is Flow network which is initialized with bootstrapped weights from 2D NN (which was trained on RGB images from ImageNet) and then trained on optical flow values from Kinetics dataset. It's suprising that flow NN is initialized with RGB values. Is it really so?

Contributor guide

Open the contributing guide

Research direction

Start with the README passage about ImageNet and Kinetics, then inspect data/checkpoints/rgb_imagenet and data/checkpoints/flow_imagenet. Determine and document how each checkpoint was initialized and trained, including whether the flow model uses bootstrapped RGB weights. Done means the README accurately explains the provenance of both checkpoints.

Written by the indexing model from the issue text.

Assessment

Domain
computer-vision, machine-learning
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.