Current limitation on transforms
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 17.9k
- Forks
- 7.3k
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 13
Description
The torchvision transforms now have 2 backends (PIL and Tensor), here are some functional mismatch between them and some may-be-useful features that neither of them support. Details are listed in transforms.py and functional.py.
Supported by PIL but not Tensor:
- Fill value for pad and random crop.
- Tensor images do not support many modes due to lack of metadata (maybe not possible to address). For instance, the adjust_* functions and autoaugment related functions.
- Tensors only support 3 interpolation modes (bilinear, linear, nearest).
- Tensors only support transformations on RGB images
Crop with crop size larger than the original image. #3297 Solved by #3333
Supported by Tensor but not PIL:
- Normalize (probably of no use for PIL images).
- Erase.
Supported by neither:
- adjust_gamma() and adjust_hue() do not support images with transparency.
- Subpixel translations. #3293
Not supported by torchscript (mostly not possible given the current jit support):
- single value inputs in Pad(fill), RandomCrop(padding), Resize(size), RandomResizedCrop(size).
- PIL and Tensor conversions.
- Compose, RandomOrder, RandomChoice.
- Lambda.
It is just a draft, let me know if I forget anything. cc @vfdev-5 @datumbox
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing torchvision/transforms/transforms.py and torchvision/transforms/functional.py, then compare the listed PIL, Tensor, and TorchScript limitations. The issue is a broad inventory rather than a scoped task, so a contributor would need to select one limitation and define its expected behavior and tests before implementation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- computer-vision
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 18/100