pytorch / pytorch/vision

[Feature Request] Datasets Should Use New `torchvision.io` Image Loader APIs and Return `TVTensor` Images by Default

Open
#8,762 9 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
17.9k
Forks
7.3k
Avg merge
1d 15h
Merged PRs (30d)
13

Description

🚀 The feature
  1. Add "torchvision" image loader backend based on new torchvision.io APIs (See: Release Notes v0.20) and enable it by default.
  2. VisionDatasets should return TVTensor images by default instead of PIL.Image.
Motivation, pitch
  1. TorchVision v0.20 introduces new torchvision.io APIs that enhance its encoding/decoding capabilities.
  2. Current VisionDatasets returns PIL.Image by default, but the first step of transforms is usually transforms.ToImage().
  3. PIL is slow (See: Pillow-SIMD), especially when compared with new torchvision.io APIs.
  4. Current TorchVision image loader backends are based on PIL or accimage, not including new torchvision.io APIs.
Alternatives
  1. The return type of datasets can be PIL.Image when using the PIL or the accimage backends, and be TVTensor if using new APIs (may lose consistency).
Additional context

I would like to make a pull request if the community likes this feature.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Read the linked torchvision v0.20 release notes first, then trace the existing PIL and accimage image-loader backends used by VisionDatasets and compare them with the torchvision.io APIs. Done means a default torchvision.io backend is enabled and VisionDatasets return TVTensor images by default, with the PIL and accimage alternatives considered for consistency.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
computer-vision, data
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.