pytorch / pytorch/vision

not all prototype transforms are serializable

Open
#6,728 0 comments 0 reactions 1 assignee View on GitHub

@pmeier is already working on this.

Since Oct 10, 2022.

bug module: transforms prototype
Dominant language
Python
Stars
17.9k
Forks
7.3k
Avg merge
1d 15h
Merged PRs (30d)
13

Description

This is important for two reasons:

  1. If we stick to torchdata for datasets v2, every object on the datapipe graph has to be serializable for multiprocessing. IIRC, this is only a requirement if forking from the main process is not possible, but that is the case on Windows.
  2. Our classification references have the ability to cache the dataset which includes the transformations:

https://github.com/pytorch/vision/blob/6e203b44098c3371689f56abc17b7c02bd51a261/references/video_classification/train.py#L189

So far we don't have a unified test framework for our prototype transforms and thus it is not easy to add tests for all transforms. Note that we don't need tests for the functional kernels and dispatchers, since they are already tested for JIT scriptability and that protocol is a lot stricter than pickling.

cc @vfdev-5 @datumbox @bjuncek

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.