NVIDIA / NVIDIA/DALI

Add an operator for receiving video metadata

Open
#5,630 5 comments 0 reactions 1 assignee View on GitHub

Nobody has claimed this yet.

enhancement Video
Dominant language
C++
Stars
5.8k
Forks
678
Avg merge
3d 1h
Merged PRs (30d)
27

Description

Is this a new feature, an improvement, or a change to existing functionality?

New Feature

How would you describe the priority of this feature request

Should have (e.g. Adoption is possible, but the performance shortcomings make the solution inferior).

Please provide a clear description of problem this feature solves

The sample rate (fps) of videos may very and hence the time period a fixed number of frames represent also varies. Having access to either the fps, duration or even the concrete timesteps of each frame is often crucial in many tasks where the actual duration in seconds is more important than the number of frames. For example, I am decoding raw video bytes from a web dataset using the experimental video decoder and I am forced to retreat to other libraries that can give me this kind of information from the raw video bytes (specifically, pytorch's VideoReader API).

Feature Description

As a user I want to be able to extract information about the sample rate of a video alongside its decoded frames.

Describe your ideal solution

A new DALI operator that extracts the desired metadata from raw video bytes.
An example video decoding pipeline reading from a webdataset (raw video bytes could also come from an external source):

@pipeline_def
def pipeline(tar_paths):
    raw_video = fn.readers.webdataset(tar_paths, ...)
    duration, fps = fn.get_video_metadata(...)
    video = fn.experimental.decoders.video(raw_video)
    return video, duration, fps
Describe any alternatives you have considered

No response

Additional context

No response

Check for duplicates
  • I have searched the open bugs/issues and have found no duplicates for this bug report

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.