GPU Test stableness tracker
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1k
- Forks
- 346
- PR merge metrics
- No merged PRs in 30d
Description
List of unstable GPU tests
-
pytorch_CycleGAN_and_pix2pix train: https://github.com/pytorch/benchmark/actions/runs/4147131894
-
DALLE2_pytorch eval (33% variation): https://github.com/pytorch/benchmark/actions/runs/4147979273
Reason: eval is dominated by memory copy

-
LearningToPaint train (115% variation): https://github.com/pytorch/benchmark/actions/runs/4287347886
Try run 1: https://github.com/pytorch/benchmark/actions/runs/4295256456 (remove CPU tensors)
Try run 2: https://github.com/pytorch/benchmark/actions/runs/4347585042 -
dense121 eval (11.2% variation):
-
detectron2_fasterrcnn_r_50_c4 eval (16% variation):
-
dlrm eval (15% variation):
-
mnasnet1_0 train (17% variation):
-
phlippe_densenet train (10% variation):
-
pytorch_CycleGAN_and_pix2pix eval (15% variation):
-
resnext50_32x4d train (10.7% variation):
-
shufflenet_v2_x1_0 eval (14.8% variation):
-
timm_vision_transformer train (16% variation): https://github.com/pytorch/benchmark/actions/runs/4147202172
Fix: https://github.com/pytorch/benchmark/pull/1419 -
Yolov3 train: https://github.com/pytorch/benchmark/actions/runs/4145521493
-
Yolov3 eval: https://github.com/pytorch/benchmark/actions/runs/4146281655
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the unchecked LearningToPaint train item and review its linked GitHub Actions runs, including the two listed attempts and the note about CPU tensors. Compare the remaining unchecked GPU benchmark tests and their reported variation. Done means the unstable tests have been investigated and the tracker items are resolved or updated with evidence.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- performance, testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100