NVIDIA / NVIDIA/DALI

VideoReader with flexible sequence_length or VFR video support

Open
#2,532 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement Video
Dominant language
C++
Stars
5.8k
Forks
678
Avg merge
3d 1h
Merged PRs (30d)
27

Description

Hi,
Upon reading the documentation, it seems each sample returned by VideoReader will contain exactly the same number of frames as defined by sequence_length. I am working on a project which requires a bit of flexibility:
In my setup, I'd like to get a number of consecutive frames in the video with different number of frames. However, the partition is known beforehand (by storing the partition timestamp/frame number in a separate text file for example).
I think this would require VideoReader to not taking a fixed number for sequence_length but rather a custom sampler implemented by the user.
The other option I could see is to support VFR video. Since in my case the partitions are evenly distributed through time (but for each unit of time, there are actually different number of frames between them).

I would first like to confirm with the current version of DALI, it is not possible to achieve what I'd like to do. And if that is the case, is there any plan for development? I would love to contribute if that's the case.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the VideoReader entry point and its documented sequence_length behavior. Confirm whether variable-length frame partitions or variable-frame-rate video are currently supported, then compare the custom sampler and VFR approaches. Done should mean that the chosen behavior is implemented and its supported usage is documented.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
data
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.