tensorflow / tensorflow/datasets
Failed to download imagenet_sketch dataset
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 4.6k
- Forks
- 1.6k
- Avg merge
- 3h 54m
- Merged PRs (30d)
- 1
Description
/!\ PLEASE INCLUDE THE FULL STACKTRACE AND CODE SNIPPET
Short description
Failed to download imagenet_sketch dataset
Environment information
tpu-vm
- Does the issue still exists with the last
tfds-nightlypackage (pip install --upgrade tfds-nightly) ?
Reproduction instructions
dataset_name="imagenet_sketch"
import tensorflow_datasets as tfds
builder = tfds.builder(dataset_name,)
builder.download_and_prepare()
Link to logs
xx@t1v-n-74b45470-w-0:/checkpoint/xx/workspace$ python3
Python 3.8.10 (default, Jun 22 2022, 20:18:18)
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
dataset_name="imagenet_sketch"
import tensorflow_datasets as tfds;builder = tfds.builder(dataset_name,);builder.download_and_prepare()
2022-10-24 07:37:56.571170: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: :/usr/local/lib
2022-10-24 07:37:56.571206: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
WARNING: Logging before InitGoogle() is written to STDERR
I0000 00:00:1666597077.762920 537548 tpu_initializer_helper.cc:94] libtpu.so already in use by another process. Run "$ sudo lsof -w /dev/accel0" to figure out which process is using the TPU. Not attempting to load libtpu.so in this process.
Downloading and preparing dataset 7.07 GiB (download: 7.07 GiB, generated: 7.61 GiB, total: 14.69 GiB) to /home/xx/tensorflow_datasets/imagenet_sketch/1.0.0...
Dl Size...: 0 MiB [00:01, ? MiB/s]████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:01<00:00, 1.17s/ url]
Dl Completed...: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:01<00:00, 1.18s/ url]
Traceback (most recent call last):
File "", line 1, in
File "/usr/local/lib/python3.8/dist-packages/tensorflow_datasets/core/dataset_builder.py", line 462, in download_and_prepare
self._download_and_prepare(
File "/usr/local/lib/python3.8/dist-packages/tensorflow_datasets/core/dataset_builder.py", line 1157, in _download_and_prepare
split_generators = self._split_generators( # pylint: disable=unexpected-keyword-arg
File "/usr/local/lib/python3.8/dist-packages/tensorflow_datasets/image_classification/imagenet_sketch/imagenet_sketch.py", line 73, in _split_generators
'archive': dl_manager.iter_archive(path),
File "/usr/local/lib/python3.8/dist-packages/tensorflow_datasets/core/download/download_manager.py", line 568, in iter_archive
return extractor.iter_archive(resource.path, resource.extract_method)
File "/usr/local/lib/python3.8/dist-packages/tensorflow_datasets/core/download/extractor.py", line 244, in iter_archive
raise ValueError(
ValueError: Cannotiter_archiveover /home/xx/tensorflow_datasets/downloads/u_0_ucid_1Mj0i5HBth_yeXzsg22gZ_export_downloadM_3TbMQ6GxLpcoS49Iutyg_3Aenh_hzbA5dQtsz0S8Q. Invalid or unrecognised archive.
Expected behavior
Expect to see no error when downloading
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the failure with the provided tfds.builder("imagenet_sketch") and download_and_prepare() snippet. Read tensorflow_datasets/image_classification/imagenet_sketch/imagenet_sketch.py, especially _split_generators, then follow download_manager.py iter_archive and extractor.py iter_archive. Done means the dataset archive is recognized and download_and_prepare() completes without the reported ValueError.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, tensorflow
- Domain
- data, machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100