Error Downloading Kinetics 400 dataset
@pmeier is already working on this.
Since Apr 26, 2022.
- Dominant language
- Python
- Stars
- 17.9k
- Forks
- 7.3k
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 13
Description
🐛 Describe the bug
Error in downloading Kinetics 400 dataset. Download stops at training set tar file 121 (full k400 dataset is 200+ tar files).
I know this isn't a problem with the disk being out of write space. This has happened twice with different users and environments.
Not seeing this exact error raised as an issue anywhere else. Seems something is wrong with one of the downloaded files? Any ideas or workaround would be greatly appreciated!
Thank you!!!!!
Code to produce Error:
import torchvision
kinetics_path = '~/kinetics400'
kinetics400_dataset = torchvision.datasets.Kinetics(root=kinetics_path,
frames_per_clip=5,
num_classes='400',
download=True)
Last part of error message (top of message is cut off, sorry.)
Traceback (most recent call last):
File "/bsdir/donwload_kinetics.py", line 4, in <module>
kinetics400_dataset = torchvision.datasets.Kinetics(root=kinetics_path,
File "/bsdir/miniconda3/envs/torch_cuda_11_3/lib/python3.9/site-packages/torchvision/datasets/kinetics.py", line 127, in __init__
self.download_and_process_videos()
File "/bsdir/miniconda3/envs/torch_cuda_11_3/lib/python3.9/site-packages/torchvision/datasets/kinetics.py", line 152, in download_and_process_videos
self._download_videos()
File "/bsdir/miniconda3/envs/torch_cuda_11_3/lib/python3.9/site-packages/torchvision/datasets/kinetics.py", line 184, in _download_videos
download_and_extract_archive(line, tar_path, self.split_folder)
File "/bsdir/miniconda3/envs/torch_cuda_11_3/lib/python3.9/site-packages/torchvision/datasets/utils.py", line 431, in download_and_extract_archive
extract_archive(archive, extract_root, remove_finished)
File "/bsdir/miniconda3/envs/torch_cuda_11_3/lib/python3.9/site-packages/torchvision/datasets/utils.py", line 408, in extract_archive
extractor(from_path, to_path, compression)
File "/bsdir/miniconda3/envs/torch_cuda_11_3/lib/python3.9/site-packages/torchvision/datasets/utils.py", line 271, in _extract_tar
with tarfile.open(from_path, f"r:{compression[1:]}" if compression else "r") as tar:
File "/bsdir/miniconda3/envs/torch_cuda_11_3/lib/python3.9/tarfile.py", line 1629, in open
return func(name, filemode, fileobj, **kwargs)
File "/bsdir/miniconda3/envs/torch_cuda_11_3/lib/python3.9/tarfile.py", line 1686, in gzopen
raise ReadError("not a gzip file")
tarfile.ReadError: not a gzip file
Versions
PyTorch version: 1.11.0
Is debug build: False
CUDA used to build PyTorch: 11.3
ROCM used to build PyTorch: N/A
OS: Ubuntu 16.04.7 LTS (x86_64)
GCC version: (Ubuntu 5.4.0-6ubuntu1~16.04.12) 5.4.0 20160609
Clang version: Could not collect
CMake version: version 3.5.1
Libc version: glibc-2.23
Python version: 3.9.7 (default, Sep 16 2021, 13:09:58) [GCC 7.5.0] (64-bit runtime)
Python platform: Linux-4.4.0-210-generic-x86_64-with-glibc2.23
Is CUDA available: True
CUDA runtime version: 9.0.176
GPU models and configuration:
GPU 0: TITAN Xp
GPU 1: TITAN Xp
GPU 2: TITAN Xp
GPU 3: TITAN Xp
GPU 4: TITAN Xp
GPU 5: TITAN Xp
Nvidia driver version: 455.45.01
cuDNN version: Probably one of the following:
/usr/lib/x86_64-linux-gnu/libcudnn.so.5.1.10
/usr/lib/x86_64-linux-gnu/libcudnn.so.6.0.21
/usr/lib/x86_64-linux-gnu/libcudnn.so.7.6.5
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True
Versions of relevant libraries:
[pip3] mypy-extensions==0.4.3
[pip3] numpy==1.21.2
[pip3] numpydoc==1.2
[pip3] steerable-pytorch==0.1
[pip3] torch==1.11.0
[pip3] torchaudio==0.11.0
[pip3] torchvision==0.12.0
[conda] blas 1.0 mkl
[conda] cudatoolkit 11.3.1 h2bc3f7f_2
[conda] libblas 3.9.0 12_linux64_mkl conda-forge
[conda] libcblas 3.9.0 12_linux64_mkl conda-forge
[conda] liblapack 3.9.0 12_linux64_mkl conda-forge
[conda] liblapacke 3.9.0 12_linux64_mkl conda-forge
[conda] mkl 2021.4.0 h06a4308_640
[conda] mkl-service 2.4.0 py39h7f8727e_0
[conda] mkl_fft 1.3.1 py39hd3c417c_0
[conda] mkl_random 1.2.2 py39h51133e4_0
[conda] mypy_extensions 0.4.3 py39h06a4308_1
[conda] numpy 1.21.2 py39h20f2e39_0
[conda] numpy-base 1.21.2 py39h79a1101_0
[conda] numpydoc 1.2 pyhd3eb1b0_0
[conda] pytorch 1.11.0 py3.9_cuda11.3_cudnn8.2.0_0 pytorch
[conda] pytorch-mutex 1.0 cuda pytorch
cc @pmeier @YosuaMichael
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.