decode_jpeg doesn't give the same result on different machine

🐛 Describe the bug

I am really confused. I am running this code on linux based machine and my M1 Mac. I am getting different results.

I verified the result from torch.frombuffer is the same on both machine. However, decode_jpeg isn't.

import numpy as np
import torch
from torchvision.io import decode_jpeg
from torchvision.transforms import ToTensor, PILToTensor
from time import time
from io import BytesIO
from PIL import Image
from lightning import seed_everything

seed_everything(42)

np_data = np.random.randint(255, size=(28, 28, 3), dtype=np.uint8)
img = Image.fromarray(np_data)

# from the JPEG image directly
path = "random_image.JPEG"
img.save(path, format="jpeg", quality=100)
img = Image.open(path)

t0 = time()
with open(path, "rb") as f:
    data = f.read()
array = torch.frombuffer(data, dtype=torch.uint8)
array_torvision = decode_jpeg(array)
print(time() - t0)

Versions

Collecting environment information...
PyTorch version: 2.0.1
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: macOS 13.5.2 (arm64)
GCC version: Could not collect
Clang version: 14.0.3 (clang-1403.0.22.14.1)
CMake version: Could not collect
Libc version: N/A

Python version: 3.9.15 (main, Dec  5 2022, 15:51:18)  [Clang 14.0.0 (clang-1400.0.29.202)] (64-bit runtime)
Python platform: macOS-13.5.2-arm64-arm-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Apple M1 Max

Versions of relevant libraries:
[pip3] mypy==1.0.0
[pip3] mypy-extensions==1.0.0
[pip3] numpy==1.24.1
[pip3] pytorch-lightning==1.9.0
[pip3] torch==2.0.1
[pip3] torchdata==0.6.1
[pip3] torchmetrics==0.11.1
[pip3] torchvision==0.15.1
[conda] Could not collect

Start by running the provided Python reproduction on both the Linux machine and the M1 Mac, recording the decoded tensor values and the relevant JPEG decoding library versions. Then trace the torchvision decode_jpeg entry point and compare the platform-specific decoding paths; done means identifying the source of the discrepancy and defining an agreed expected result.