pytorch / pytorch/vision

Add more docs about how to build a wheel of vision with the all features of video

Open
#7,250 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

module: documentation module: video
Dominant language
Python
Stars
17.9k
Forks
7.3k
Avg merge
1d 15h
Merged PRs (30d)
13

Description

🚀 The feature

No docs to show how to build a wheel with the all features of video including the video_reader(gpu decoder).

Motivation, pitch

I want to use GPU to accelerate the speed of video decoding.
And i find that you support the gpu video decoder.
There are some questions below:

  1. from https://github.com/pytorch/vision#video-backend, I know that i need ffmpeg or pyav to enable the video feature. However, both of them do not support GPU originally. So what do i need if i want to use GPU video decoder.
  2. No detail docs to show how to build a wheel of vision with GPU video decoder.
  3. After gpu decoding,where is the tensor, system memory or gpu memory?
  4. What's the data flow of your video processing and inference?
1. Decoding in the gpu memory
2. Downloading to the system memory.
3. Uploading to the gpu memory for inference.
4. Downloading to the system memory.
5. Uploading to gpu memory for encoding.(Maybe it does not exist)

or

1. Decoding in the gpu memory
2. Inference in the gpu memory directly.
3. Encoding in the gpu memory(Maybe it does not exist)

5.Is there any way for video to work with this pipeline——1.decoded by gpu and keep it in the gpu memory. 2.Inference with tensor in gpu memory directly without downloading to the system memory and uploading to gpu memory for inference again.

I think you should add these to docs.

Alternatives

No response

Additional context

No response

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the repository's video-backend documentation and the video_reader GPU decoder mentioned in the issue. Determine the supported GPU-decoding prerequisites and wheel build process, then document tensor placement and the video processing and inference data flow, including whether GPU-resident inference is supported.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
computer-vision, documentation
Issue type
Documentation
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.