pytorch / pytorch/vision

Padding transform to a multiple of 2

Open
#4,865 2 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
17.9k
Forks
7.3k
Avg merge
1d 15h
Merged PRs (30d)
13

Description

🚀 The feature

Introduce a new API to pad a tensor, such as transforms.PadSizeDivisor (I don't come up with a proper name at the moment so that it could be changed)

Expectation:

inputs = torch.rand(1,3,32,254)
tf = transforms.PadSizeDivisor(size_divisor=32, dim=3, pad_method='constant', pad_value=0)  # maybe we can support dim=[2,3]?
outputs = tf(inputs)
assert outputs.shape == torch.Size([1,3,32,256])
Motivation, pitch

I think padding with the size_divisor argument is widely used in segmentation or U-like network where we apply downsampling followed by upsampling the input tensor. In most cases, the up/down ratio is multiple 2, e.g. 2, 4, 8, 16, 32. Therefore, the input is usually padded to some sort of these multiples such as 352, 768 in spatial dimensions (H, W).

The current implementation of transforms.Pad only supports fixed padding size. It works great during training where we usually apply the tensor crop or resize to a fixed size to the input tensor. However, during inference, we usually maintain the ratio when resizing. This leads to the remaining edge is not guaranteed to be a multiple of 2, and thus, the following up/down operators would be failed due to the size mismatch. At the moment, the solution is usually to compute the padding size on each edge before or after resizing the input image.

Therefore, I think it would be great to have a library-included transformation so that we can reuse it in our application.

Alternatives

No response

Additional context

No response

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading the existing transforms.Pad API and compare it with the tensor-shape example in the issue. Define the new transform's behavior for size_divisor, dim, pad_method, and pad_value, including whether multiple dimensions are supported; it is done when the example produces a width of 256.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
computer-vision
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.