[feature request] Make `transforms.functional_tensor` functions differential w.r.t. their parameters
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 17.9k
- Forks
- 7.3k
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 13
Description
🚀 The feature
Make operations in torchvision.transforms.functional_tensor differential w.r.t. hyper-parameters, which is helpful for Faster AutoAugment search (hyper-parameters are learnable parameters via backward). (while keeping the backward compatibility to previous codes)
Some operations are not differential (e.g., Posterize), which might require users to write their own implementations.
Motivation, pitch
The main motivation is for research purpose. Faster Autoaugment proposes to search for augment architectures using a DARTS-like framework, and all magnitudes and weights are trainable parameters. This requires all operations to have gradients w.r.t. magnitudes. This idea provides a faster search strategy as state-of-the-art AutoAugment policy search algorithms.
This work has been maintained by autoalbument and applied on some industrial scenarios from their document claims.
I think adding the backward feature wrt magnitudes would be more convenient and support future research as well.
Alternatives
No response
Additional context
Linked PR: #4995
cc @vfdev-5 @datumbox
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in torchvision.transforms.functional_tensor and review the operations whose hyper-parameters need gradients, using the Faster AutoAugment motivation and linked PR #4995 as context. Preserve backward compatibility with existing code; done means supported operations are differentiable with respect to their magnitudes while acknowledging operations such as Posterize may remain unsupported.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- computer-vision
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100