pytorch / pytorch/vision

Unit tests for Vision

Open
#110 8 comments 3 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

help wanted
Dominant language
Python
Stars
17.9k
Forks
7.3k
Avg merge
1d 15h
Merged PRs (30d)
13

Description

This issue fleshes out the full details and scope of the unit tests needed for torchvision.

There are very limited unit tests under test/ which dont cover the transform outputs themselves, but are limited to dimension and shape checks.

First, let's start with quantitative tests on known results.

We need to have a set of 10 test images, and then do each transformation of vision.transforms on these 10 images, and compare them pixel-wise with known results. We then compare the known results with the computed result from the transforms, and if they are within some threshold, we pass the test.

Some of the transforms such as Horizontal / Vertical flip can also have exact numerical unit-tests.

The test images:

  • 2 monochrome images
  • 2 3-channel images
  • 2 4-channel PNG images with an Alpha component

Can find some on Wikipedia that are freely licensed.

The tests need to cover all transforms under: https://github.com/pytorch/vision#transforms

For similar testing, you can have a look at:

https://github.com/torch/image/blob/master/test/test.lua#L258-L646

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading the existing tests under test/ and the transforms listed in the torchvision README. Review the referenced torch/image tests for comparison patterns, then define the freely licensed test-image set and expected results. Done means every vision.transforms transform has quantitative coverage, including exact checks where applicable.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
computer-vision, testing-qa
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.