Restructuring torchvision tests
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 17.9k
- Forks
- 7.3k
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 13
Description
Motivation
Our tests are getting a bit lengthy and considering we are adding new features, we are actually placing multiple tests in single file.
This leads to problems as the tests become harder to refactor and harder to read, manage. Standalone files for tests or grouped according to purpose are easier to anaylse and debug as well.
Our longest test files
Copy paste-able command
find test -name "*.py" | xargs -I {} wc -l {} | sort -nr | head -n 10
oke@psycon:~/Aditya/PyTorch/vision$ find test -name "*.py" | xargs -I {} wc -l {} | sort -nr | head -n 10
2675 test/test_datasets.py
2242 test/test_transforms.py
1535 test/builtin_dataset_mocks.py
1528 test/test_ops.py
1343 test/test_functional_tensor.py
1339 test/test_prototype_transforms_functional.py
1254 test/test_video_reader.py
982 test/test_transforms_tensor.py
954 test/datasets_utils.py
942 test/test_models.py
Many splits are possible that can make the tests smaller and easier to manage
E.g.
test_models.py
----- test_classification_models.py
----- test_detection_models.py
----- test_video_models.py
test_ops.py
------ test_post_processors.py
------ test_losses.py
------ test_layers.py
Possibilities are many, we can have multiple files with better grouping.
Pitch
We should think of grouping tests in logically similar way together. or consider creating test folder a package. See
https://github.com/pytorch/vision/pull/4436 . Earlier we have faced problems doing the same.
From discussion with Phillip offline
Considering we don't Pin pytest. We can make use of https://github.com/pytest-dev/pytest/pull/9134
With the python path param, we can probably make use of common_utils.py file as well. So that we can test files without hurting other scenarios which were mentioned in the comments.
Alternatives
This is quite tricky but very helpful as we don't want tests to hurt us later.
Reference
https://github.com/pytorch/vision/pull/4436#issuecomment-923131237
https://github.com/pytorch/vision/pull/4436#issuecomment-924005657
https://github.com/pytorch/vision/pull/4436#issuecomment-924250093
cc @NicolasHug @datumbox @pmeier
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by running the provided find test command and reviewing the longest files, including test/test_datasets.py, test/test_transforms.py, test/test_ops.py, and test/test_models.py. Read the referenced pull request discussions, especially PR 4436, before choosing a grouping. Done means tests are logically split into manageable files while preserving behavior and avoiding the earlier test-package problems.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- testing-qa
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100