pytorch / pytorch/tutorials

[BUG] - Incorrect ResNet18 preprocessing in Captum tutorial

Open Beginner friendly
#3,936 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
9.3k
Forks
4.4k
Avg merge
1d 21h
Merged PRs (30d)
4

Description

Add Link

https://docs.pytorch.org/tutorials/beginner/introyt/captumyt.html

Describe the bug

The Captum tutorial uses the following preprocessing for a pretrained ResNet18 model:

transform = transforms.Compose([
    transforms.Resize(224),
    transforms.CenterCrop(224),
    transforms.ToTensor(),
])

transform_normalize = transforms.Normalize(
    mean=[0.485, 0.456, 0.406],
    std=[0.229, 0.224, 0.225],
)

However, the preprocessing associated with
ResNet18_Weights.IMAGENET1K_V1 uses a resize size of 256 followed by a
center crop of 224:

weights = models.ResNet18_Weights.IMAGENET1K_V1
preprocess = weights.transforms()

print(preprocess)

Output:

ImageClassification(
    crop_size=[224]
    resize_size=[256]
    mean=[0.485, 0.456, 0.406]
    std=[0.229, 0.224, 0.225]
    interpolation=InterpolationMode.BILINEAR
)

Resize(224) and Resize(256) -> CenterCrop(224) do not produce the
same model input. Resizing the shorter side directly to 224 changes the
scale and visible image region compared with the preprocessing associated
with the selected pretrained weights.

For my test image, the top-class predicted probability was approximately
95% with the tutorial preprocessing and 98% with
ResNet18_Weights.IMAGENET1K_V1.transforms().

This is a difference in confidence for one image, not a measurement of
dataset-level accuracy. However, it demonstrates that the current
preprocessing materially changes the model output and may also change the
resulting Captum attribution maps.

Sample code to reproduce

import torch
from torchvision import models, transforms

weights = models.ResNet18_Weights.IMAGENET1K_V1
model = models.resnet18(weights=weights).eval()

tutorial_preprocess = transforms.Compose([
    transforms.Resize(224),
    transforms.CenterCrop(224),
    transforms.ToTensor(),
    transforms.Normalize(
        mean=[0.485, 0.456, 0.406],
        std=[0.229, 0.224, 0.225],
    ),
])

official_preprocess = weights.transforms()

x_tutorial = tutorial_preprocess(test_img).unsqueeze(0)
x_official = official_preprocess(test_img).unsqueeze(0)

with torch.inference_mode():
    tutorial_probabilities = model(x_tutorial).softmax(dim=1)
    official_probabilities = model(x_official).softmax(dim=1)

class_id = official_probabilities.argmax(dim=1).item()

print(
    "Tutorial preprocessing:",
    tutorial_probabilities[0, class_id].item(),
)
print(
    "Official preprocessing:",
    official_probabilities[0, class_id].item(),
)
print(
    "Maximum input difference:",
    (x_tutorial - x_official).abs().max().item(),
)

Expected Result:

The tutorial preprocessing should match the preprocessing associated with
the pretrained weights. The tutorial could either use
weights.transforms() or change Resize(224) to Resize(256) while
retaining the separate normalization step needed by the tutorial.

Actual Result:

The tutorial resizes the shorter image side to 224 instead of 256. This
produces a different input tensor, changes the predicted probabilities,
and may change the Captum attribution visualization.

No exception or traceback is produced. This is a preprocessing correctness
issue.

Describe your environment

Platform: Windows-11-10.0.26200-SP0
PyTorch: 2.13.0+cu132
Torchvision: 0.28.0+cu132
CUDA available: True
CUDA version: 13.2

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the Captum tutorial at https://docs.pytorch.org/tutorials/beginner/introyt/captumyt.html and inspect its ResNet18 preprocessing. Compare the tutorial transform with ResNet18_Weights.IMAGENET1K_V1.transforms() and update the preprocessing so it matches the selected pretrained weights. Done means the tutorial uses the expected 256 resize and 224 center crop, with the sample reproduction showing matching input preprocessing.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
76/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.