`affine` creates artefacts on the edges of the image

🐛 Describe the bug

When employing the affine functional operation (in both v1 and v2), it's evident that black borders are introduced around the image, even when the fill value matches the image content. These black margins are observable when using both uint8 and float32 data types, and this phenomenon occurs consistently on both Ubuntu and Mac M1.

Upon comparing the implementation of the 'affine' operation in torchvision with that in Kornia, I am uncertain whether the interpolation issue is limited to the image edges. Notably, when utilizing Kornia, the output appears to be more visually appealing when applied to an image.

import torch
import torchvision
from torchvision.transforms.v2.functional import affine
from torchvision.tv_tensors import Image
from torchvision.transforms.v2.functional._geometry import _get_inverse_affine_matrix
from kornia.geometry.transform import get_affine_matrix2d, warp_affine
from torchvision.transforms import InterpolationMode

image = Image(128 * torch.ones((3, 240, 200), dtype=torch.float))

angle =30
trans = (0,0)
scale = 1.0
shear = (0,0)
center = (image.shape[-1] / 2, image.shape[-2] / 2)
inter = InterpolationMode.BILINEAR
fill = [128, 128, 128]

M = get_affine_matrix2d(
    torch.Tensor(trans),
    torch.Tensor([center]),
    torch.Tensor([[scale, scale]]),
    torch.Tensor([angle]),
    torch.Tensor([shear[0]]),
    torch.Tensor([shear[1]]),
)

kn_img = warp_affine(
    image.unsqueeze(0),
    M[:, :2],
    image.shape[-2:],
    mode="bilinear",
    padding_mode="fill",
    fill_value=torch.tensor(fill),
    align_corners=False,
)

tv_img = affine(
    image,
    angle=angle,
    translate=trans,
    scale=scale,
    shear=shear,
    fill=fill,
    interpolation=inter,
)

torchvision.io.write_png(kn_img[0].to(dtype=torch.uint8), "affine_kornia.png")
torchvision.io.write_png(tv_img.to(dtype=torch.uint8), "affine_torchvision.png")

Kornia	Torchvision

Versions

PyTorch version: 2.1.0
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: macOS 13.6 (arm64)
GCC version: Could not collect
Clang version: 15.0.0 (clang-1500.0.40.1)
CMake version: Could not collect
Libc version: N/A

Python version: 3.10.9 (main, Jun 29 2023, 12:23:23) [Clang 14.0.3 (clang-1403.0.22.14.1)] (64-bit runtime)
Python platform: macOS-13.6-arm64-arm-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Apple M1 Pro

Versions of relevant libraries:
[pip3] flake8==6.1.0
[pip3] mypy==1.6.0
[pip3] mypy-extensions==1.0.0
[pip3] numpy==1.25.2
[pip3] onnx==1.14.1
[pip3] pytorch-lightning==2.0.9
[pip3] pytorch-ranger==0.1.1
[pip3] torch==2.1.0
[pip3] torch-optimizer==0.3.0
[pip3] torchdata==0.6.1
[pip3] torchmetrics==1.0.3
[pip3] torchtext==0.15.2
[pip3] torchvision==0.16.0

cc @vfdev-5

Start with torchvision.transforms.v2.functional.affine and the imported _get_inverse_affine_matrix in torchvision.transforms.v2.functional._geometry. Run the provided constant-image reproduction for both uint8 and float32 inputs, then compare the edge output with the reported Kornia result. Done means affine no longer introduces black edge artefacts when the fill value matches the image content, with coverage for the affected cases.