meta-pytorch / meta-pytorch/torchcodec

Add cap resize to the Resize

Open
#1,493 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1.2k
Forks
125
Avg merge
22h 47m
Merged PRs (30d)
54

Description

🚀 The feature

The torchcodec.transforms.Resize's aiming is https://meta-pytorch.org/torchcodec/stable/generated_examples/decoding/performance_tips.html#decoder-native-transforms

TorchCodec supports applying transforms like resize and crop during the decoding process itself, rather than as a separate post-processing step. This can lead to significant memory savings, especially when decoding high-resolution videos that will be resized to smaller dimensions.

But if we want to be able to save memory we currently have to:

decoder = VideoDecoder("file.mp4")
w, h = decoder.metadata.width, decoder.metadata.height
if (longer := max(w, h)) > max_size:
    scale = max_size / longer
    decoder = VideoDecoder("file.mp4", transforms=[Resize(size=(w*scale, h*scale))]
)
...

This may cause memory leak since VideoDecoder is double created and it has no excplit .close(); and it adds lots of boilerplate code.

Except:
We can have VideoDecoder("file.mp4", max_frame_size...) or decoder = VideoDecoder("file.mp4", transforms=[Resize(maxsize=...)] to avoid create twice VideoDecoder

This will also make it good to local comsumer development envioments.

Motivation, pitch

No response

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with torchcodec.transforms.Resize and the VideoDecoder construction path, then read the linked decoder-native transforms documentation. Compare the proposed max-size behavior with the existing transform API and determine how resizing can be selected during decoder creation without constructing VideoDecoder twice. Done means a documented cap-resize API that applies during decoding and preserves expected dimensions.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
audio-video-rtc, performance
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.