Discrepancy in output of torchvision.io.read_image vs PIL.Image

🐛 Describe the bug

Some images from the imagenetv2 dataset (downloadable here) contain nonzero differences when loaded using torchvision.io.read_image, with some images containing large differences in pixel values.

import torch
import torchvision.io
import numpy as np
from PIL import Image

def loadimage_pil(path):
    return torch.tensor(np.array(Image.open(path).convert("RGB"))).permute(2, 0, 1)

def loadimage_torchio(path):
    return torchvision.io.read_image(path, torchvision.io.ImageReadMode.RGB)

# assuming archive is unpacked in the same folder as script - change accordingly.
filepath = "./imagenetv2-matched-frequency-format-val/455/aaaf43c110a10aabce09700a6a3cfb2622b4847a.jpeg"
print(f"loading '{filepath}'")
img_pil = loadimage_pil(filepath)
img_tio = loadimage_torchio(filepath)
difference = img_pil.to(float) - img_tio.to(float)

error = torch.sqrt(torch.mean(torch.square(difference)))
print(error)
# > tensor(6.3985, dtype=torch.float64)

When loading the file used in the example imagenetv2-matched-frequency-format-val/455/aaaf43c110a10aabce09700a6a3cfb2622b4847a.jpeg the printed error value is 6.3985.

Versions

Collecting environment information...
PyTorch version: 2.1.0
Is debug build: False
CUDA used to build PyTorch: 11.8
ROCM used to build PyTorch: N/A

OS: Fedora Linux 36 (Thirty Six) (x86_64)
GCC version: (GCC) 12.2.1 20220819 (Red Hat 12.2.1-2)
Clang version: Could not collect
CMake version: Could not collect
Libc version: glibc-2.35

Python version: 3.10.8 | packaged by conda-forge | (main, Nov 22 2022, 08:26:04) [GCC 10.4.0] (64-bit runtime)
Python platform: Linux-6.0.8-200.fc36.x86_64-x86_64-with-glibc2.35
Is CUDA available: True
CUDA runtime version: 11.7.99
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration: 
<snip>

Nvidia driver version: 520.56.06
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True


Versions of relevant libraries:
[pip3] mypy==1.2.0
[pip3] mypy-extensions==1.0.0
[pip3] numpy==1.24.4
[pip3] torch==2.1.0
[pip3] torch-tb-profiler==0.4.3
[pip3] torchaudio==2.1.0
[pip3] torchinfo==1.8.0
[pip3] torchvision==0.16.0
[pip3] triton==2.1.0
[conda] blas                      1.0                         mkl    conda-forge
[conda] libblas                   3.9.0            16_linux64_mkl    conda-forge
[conda] libcblas                  3.9.0            16_linux64_mkl    conda-forge
[conda] libjpeg-turbo             2.0.0                h9bf148f_0    pytorch
[conda] liblapack                 3.9.0            16_linux64_mkl    conda-forge
[conda] liblapacke                3.9.0            16_linux64_mkl    conda-forge
[conda] mkl                       2022.1.0           h84fe81f_915    conda-forge
[conda] numpy                     1.24.4                   pypi_0    pypi
[conda] pytorch                   2.1.0           py3.10_cuda11.8_cudnn8.7.0_0    pytorch
[conda] pytorch-cuda              11.8                 h7e8668a_5    pytorch
[conda] pytorch-mutex             1.0                        cuda    pytorch
[conda] torch-tb-profiler         0.4.3                    pypi_0    pypi
[conda] torchaudio                2.1.0               py310_cu118    pytorch
[conda] torchinfo                 1.8.0              pyhd8ed1ab_0    conda-forge
[conda] torchtriton               2.1.0                     py310    pytorch
[conda] torchvision               0.16.0              py310_cu118    pytorch

Start by reproducing the discrepancy with torchvision.io.read_image and PIL.Image on the linked ImageNetV2 JPEG, using ImageReadMode.RGB and the example comparison. Trace the two loading paths to determine why pixel values differ, then verify that the chosen behavior is consistent on the reported file and similar images.