pytorch / pytorch/vision

Container for multi-pass and randomized subset transform loops

Open
#9,532 6 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
17.9k
Forks
7.3k
Avg merge
1d 15h
Merged PRs (30d)
13

Description

🚀 The feature

I propose adding a new meta-transform container to the v2 ecosystem. This container would accept a list of child transforms and execute a randomly sampled subset of them over a configurable number of successive passes, with randomized parameters applied at each step.

The container could take configuration arguments to dynamically shuffle and repeat operations from a pool of provided transforms.

Motivation, pitch

I am working on low-level image restoration and super-resolution pipelines. To train my models, I need to implement high-order degradation pipelines, similar to the double-pass loops pioneered by Real-ESRGAN in 2021. This process requires shuffling the execution order of operations and repeating a sequence of transformations across multiple successive loops to mimic complex, layered artifacts.

Currently, torchvision.transforms.v2 does not seem to have a container to execute this natively. While RandomOrder exists, it appears to enforce a single pass where every provided transform must run exactly once. I have not found a straightforward way to use it for dynamic multi-pass loops, subset sampling, or repeated execution paths.

Because of this limitation, it feels completely necessary to bypass the transform ecosystem for these workflows. Even when using wrap_dataset_for_transforms_v2 to get structured tv_tensors, I might have to manually unpack the data, handcode custom execution loops, and manage randomization myself. A native container could cleanly solve this and keep these workflows entirely within the v2 pipeline.

Alternatives

Nesting Existing Containers: I considered manually stacking combinations of RandomOrder, RandomChoice, and Compose. However this seems to create rigid blocks that may not easily handle dynamic multi-pass subsets or varying parameter ranges across different loop passes.

Custom Transforms: Writing a massive and monolithic custom transform wrapper for the entire pipeline, but this defeats the purpose of modular, reusable transform objects.

External Libraries: Moving away from Torchvision to third-party augmentation packages which fragments the codebase.

Additional context

This addition would provide a reusable structural utility for workflows involving complex data synthesis.

from torchvision.transforms import v2

pipeline = v2.RandomSequence(
    [
        v2.GaussianBlur(kernel_size=(7, 21)),
        v2.GaussianBlur(kernel_size=(3, 11)),
        v2.JPEGCompression(quality=(30, 95)),
        v2.GaussianNoise(sigma=(0.0, 0.2))
    ],
    num_passes=2,
    num_ops_per_pass=2
)

If this fits the roadmap of the v2 framework, I would be happy to implement this container and open a pull request for it.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading the v2.RandomOrder, RandomChoice, and Compose containers to compare their execution semantics. Define how configurable passes, subset sampling, shuffling, repetition, and per-step random parameters should interact. Done means the proposed container has an agreed API and behavior that supports the multi-pass image-degradation workflow described here.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
computer-vision, machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.