torchvision.transforms cannot handle certain transformations (ResNet example)
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 17.9k
- Forks
- 7.3k
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 13
Description
I'm trying to replicate the ResNet paper, so I need the following transforms
- Subtract per-pixel-mean from all pixels in all channels (this can lead to negative values) I can do this with
Lambda Pad(4)RandomHorizontalFlip()RandomCrop([32, 32])ToTensor()
The problem is that transforms 2-4 only work on PIL.Image, while transform 1 must happen on np.array or torch.Tensor because of the negative values. I can't convert back and forth inside of the pipeline since that would cut off the negative values, so I'm stuck.
It would be great if there were versions of at least the spatial transforms which work on numpy arrays. As far as I can see, some of them convert the PIL image to array anyway (e.g. Pad).
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the torchvision.transforms pipeline around Lambda, Pad, RandomHorizontalFlip, RandomCrop, and ToTensor, focusing on their PIL Image assumptions. Reproduce the ResNet transform sequence and determine which spatial transforms need array or tensor support; done means the pipeline can preserve negative values without converting back through PIL.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- numpy, python, pytorch
- Domain
- computer-vision, machine-learning
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100