NVIDIA / NVIDIA/DALI

Videos with various length and fps

Open
#741 14 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

question Video
Dominant language
C++
Stars
5.8k
Forks
678
Avg merge
3d 1h
Merged PRs (30d)
27

Description

Hi,
Thanks for the nice library. I found DALI while looking for a video loader for action recognition. I found that DALI yet cannot handle various resolution as in the issue #725 which is necessary for public dataset such as Kinetics.

Another necessary component might be processing videos with various length and fps.
It seems VideoReader only support extract whole video into batch of sequence_length of sequences. I'm not sure because I've just tested video_test.py only.

  1. I wonder if it is possible to randomly extract one short "clip" (sequence of length sequence_length and step) from one video. It seems that this way is commonly used in training phase of Kinetics dataset.
    For evaluation, people often extract several clips along the whole video with equal interval.
  2. Additionally it will be nice if we can set the fps of all video same since videos vary in fps. I often use ffmpeg with fps filter when I extract the frames manually.

Hopefully, those process can be possible already or do you have any plan to support those features?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the VideoReader entry point and the video_test.py example to determine the current sequence extraction behavior. Review how the requested random or interval-based clips and common FPS would fit the existing API, including the mentioned ffmpeg fps-filter workflow. Done would mean clearly specified support for variable-length/FPS videos and tests covering the requested extraction modes.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, python
Domain
data, machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.