aws / aws/amazon-sagemaker-examples

[Bug Report] RuntimeError: Dataset not found. You can use download=True to download it for pytorch minist horovod

Open
#4,566 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
11k
Forks
7k
Avg merge
8h 29m
Merged PRs (30d)
8

Description

I got `RuntimeError: Dataset not found. You can use download=True ` when running [MNIST Training using PyTorch](https://sagemaker-examples.readthedocs.io/en/latest/sagemaker-python-sdk/pytorch_horovod_mnist/pytorch_mnist_horovod.html).

I am trying to change the Python version to 3.8 as a workaround but got new error:
```
ErrorMessage "RuntimeError: CUDA error: an illegal memory access was encountered
CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1."
```

Any hints appreciate!

Contributor guide

Open the contributing guide

Research direction

Start with the linked MNIST Training using PyTorch example at sagemaker-python-sdk/pytorch_horovod_mnist/pytorch_mnist_horovod.html and reproduce the reported dataset error. Compare the original run with the Python 3.8 attempt and CUDA error, then document the confirmed cause and a reproducible resolution for the example.

Written by the indexing model from the issue text.

Assessment

Tech stack
jupyter-notebook, python, pytorch
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.