aws / aws/amazon-sagemaker-examples
Pytorch MNIST sample claims to be multi-GPU example, but it's not
Open
bug:distributed-training
related: PyTorch
status: awaiting response
type: question
- Dominant language
- Jupyter Notebook
- Stars
- 11k
- Forks
- 7k
- Avg merge
- 8h 29m
- Merged PRs (30d)
- 8
Description
See script here: https://github.com/awslabs/amazon-sagemaker-examples/blob/master/sagemaker-python-sdk/pytorch_mnist/mnist.py#L109
The script is missing starting via multiprocessing.spawn method multiple training processes (1 per GPU). Right now script script is working only for single GPU.
Contributor guide
Research direction
Start by reading sagemaker-python-sdk/pytorch_mnist/mnist.py around line 109 and compare the sample's current training startup with PyTorch multiprocessing.spawn usage. Run the example on multiple GPUs and verify that it starts one training process per GPU rather than operating only on a single GPU.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- distributed-systems, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 35/100