aws / aws/amazon-sagemaker-examples
[Bug Report] RuntimeError: Dataset not found. You can use download=True to download it for pytorch minist horovod
- Dominant language
- Jupyter Notebook
- Stars
- 11k
- Forks
- 7k
- Avg merge
- 8h 29m
- Merged PRs (30d)
- 8
Description
I got `RuntimeError: Dataset not found. You can use download=True ` when running [MNIST Training using PyTorch](https://sagemaker-examples.readthedocs.io/en/latest/sagemaker-python-sdk/pytorch_horovod_mnist/pytorch_mnist_horovod.html).
I am trying to change the Python version to 3.8 as a workaround but got new error:
```
ErrorMessage "RuntimeError: CUDA error: an illegal memory access was encountered
CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1."
```
Any hints appreciate!
Contributor guide
Research direction
Start with the linked MNIST Training using PyTorch example at sagemaker-python-sdk/pytorch_horovod_mnist/pytorch_mnist_horovod.html and reproduce the reported dataset error. Compare the original run with the Python 3.8 attempt and CUDA error, then document the confirmed cause and a reproducible resolution for the example.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- jupyter-notebook, python, pytorch
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100