tensorflow / tensorflow/datasets

Can't download "seleb_a"

Open
#3,855 2 comments 3 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
4.6k
Forks
1.6k
Avg merge
3h 54m
Merged PRs (30d)
1

Description

/!\ PLEASE INCLUDE THE FULL STACKTRACE AND CODE SNIPPET

Short description
Can't download "seleb_a"

Environment information

  • Operating System: <Ubuntu 20.04 focal (x86-64)>

  • Python version: <3.8.10>

  • tensorflow-datasets/tfds-nightly version: <4.5.2+nightly>

  • tensorflow/tf-nightly version: <2.8.0>

  • Does the issue still exists with the last tfds-nightly package (pip install --upgrade tfds-nightly) ?
    YES
    Reproduction instructions


import tensorflow_datasets as tfds

celeba_bldr = tfds.builder('celeb_a')
print(celeba_bldr.info)
celeba_bldr.download_and_prepare()

If you share a colab, make sure to update the permissions to share it.

Link to logs
If applicable, <link to gist with logs, stack trace>

Expected behavior
I need celeb_a dataset

Additional context
Output: build_celeba.py

2022-03-23 18:57:25.652770: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory
2022-03-23 18:57:25.652786: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
2022-03-23 18:57:26.961914: W tensorflow/core/platform/cloud/google_auth_provider.cc:184] All attempts to get a Google authentication bearer token failed, returning an empty token. Retrieving token from files failed with "NOT_FOUND: Could not locate the credentials file.". Retrieving token from GCE failed with "FAILED_PRECONDITION: Error executing an HTTP request: libcurl code 6 meaning 'Couldn't resolve host name', error details: Could not resolve host: metadata".
tfds.core.DatasetInfo(
name='celeb_a',
full_name='celeb_a/2.0.1',
description="""
CelebFaces Attributes Dataset (CelebA) is a large-scale face attributes dataset with more than 200K celebrity images, each with 40 attribute annotations. The images in this dataset cover large pose variations and background clutter. CelebA has large diversities, large quantities, and rich annotations, including
- 10,177 number of identities,
- 202,599 number of face images, and
- 5 landmark locations, 40 binary attributes annotations per image.

The dataset can be employed as the training and test sets for the following computer vision tasks: face attribute recognition, face detection, and landmark (or facial part) localization.

Note: CelebA dataset may contain potential bias. The fairness indicators
[example](https://www.tensorflow.org/responsible_ai/fairness_indicators/tutorials/Fairness_Indicators_TFCO_CelebA_Case_Study)
goes into detail about several considerations to keep in mind while using the
CelebA dataset.
""",
homepage='http://mmlab.ie.cuhk.edu.hk/projects/CelebA.html',
data_path='~/tensorflow_datasets/celeb_a/2.0.1',
download_size=1.38 GiB,
dataset_size=1.62 GiB,
features=FeaturesDict({
    'attributes': FeaturesDict({
        '5_o_Clock_Shadow': tf.bool,
        'Arched_Eyebrows': tf.bool,
        'Attractive': tf.bool,
        'Bags_Under_Eyes': tf.bool,
        'Bald': tf.bool,
        'Bangs': tf.bool,
        'Big_Lips': tf.bool,
        'Big_Nose': tf.bool,
        'Black_Hair': tf.bool,
        'Blond_Hair': tf.bool,
        'Blurry': tf.bool,
        'Brown_Hair': tf.bool,
        'Bushy_Eyebrows': tf.bool,
        'Chubby': tf.bool,
        'Double_Chin': tf.bool,
        'Eyeglasses': tf.bool,
        'Goatee': tf.bool,
        'Gray_Hair': tf.bool,
        'Heavy_Makeup': tf.bool,
        'High_Cheekbones': tf.bool,
        'Male': tf.bool,
        'Mouth_Slightly_Open': tf.bool,
        'Mustache': tf.bool,
        'Narrow_Eyes': tf.bool,
        'No_Beard': tf.bool,
        'Oval_Face': tf.bool,
        'Pale_Skin': tf.bool,
        'Pointy_Nose': tf.bool,
        'Receding_Hairline': tf.bool,
        'Rosy_Cheeks': tf.bool,
        'Sideburns': tf.bool,
        'Smiling': tf.bool,
        'Straight_Hair': tf.bool,
        'Wavy_Hair': tf.bool,
        'Wearing_Earrings': tf.bool,
        'Wearing_Hat': tf.bool,
        'Wearing_Lipstick': tf.bool,
        'Wearing_Necklace': tf.bool,
        'Wearing_Necktie': tf.bool,
        'Young': tf.bool,
    }),
    'image': Image(shape=(218, 178, 3), dtype=tf.uint8),
    'landmarks': FeaturesDict({
        'lefteye_x': tf.int64,
        'lefteye_y': tf.int64,
        'leftmouth_x': tf.int64,
        'leftmouth_y': tf.int64,
        'nose_x': tf.int64,
        'nose_y': tf.int64,
        'righteye_x': tf.int64,
        'righteye_y': tf.int64,
        'rightmouth_x': tf.int64,
        'rightmouth_y': tf.int64,
    }),
}),
supervised_keys=None,
disable_shuffling=False,
splits={
    'test': <SplitInfo num_examples=19962, num_shards=2>,
    'train': <SplitInfo num_examples=162770, num_shards=16>,
    'validation': <SplitInfo num_examples=19867, num_shards=2>,
},
citation="""@inproceedings{conf/iccv/LiuLWT15,
  added-at = {2018-10-09T00:00:00.000+0200},
  author = {Liu, Ziwei and Luo, Ping and Wang, Xiaogang and Tang, Xiaoou},
  biburl = {https://www.bibsonomy.org/bibtex/250e4959be61db325d2f02c1d8cd7bfbb/dblp},
  booktitle = {ICCV},
  crossref = {conf/iccv/2015},
  ee = {http://doi.ieeecomputersociety.org/10.1109/ICCV.2015.425},
  interhash = {3f735aaa11957e73914bbe2ca9d5e702},
  intrahash = {50e4959be61db325d2f02c1d8cd7bfbb},
  isbn = {978-1-4673-8391-2},
  keywords = {dblp},
  pages = {3730-3738},
  publisher = {IEEE Computer Society},
  timestamp = {2018-10-11T11:43:28.000+0200},
  title = {Deep Learning Face Attributes in the Wild.},
  url = {http://dblp.uni-trier.de/db/conf/iccv/iccv2015.html#LiuLWT15},
  year = 2015
}""",

)
Downloading and preparing dataset 1.38 GiB (download: 1.38 GiB, generated: 1.62 GiB, total: 3.00 GiB) to ~/tensorflow_datasets/celeb_a/2.0.1...
Dl Size...: 0 MiB [00:00, ? MiB/s] | 0/4 [00:00<?, ? url/s]
Dl Completed...: 0%| | 0/4 [00:00<?, ? url/s]
Traceback (most recent call last):
File "/media/a1/SW/python-machine-learning-book-3rd-edition/ch15/build_celeba.py", line 5, in
celeba_bldr.download_and_prepare()
File "/home/felixt/.local/lib/python3.8/site-packages/tensorflow_datasets/core/dataset_builder.py", line 461, in download_and_prepare
self._download_and_prepare(
File "/home/felixt/.local/lib/python3.8/site-packages/tensorflow_datasets/core/dataset_builder.py", line 1146, in _download_and_prepare
split_generators = self._split_generators( # pylint: disable=unexpected-keyword-arg
File "/home/felixt/.local/lib/python3.8/site-packages/tensorflow_datasets/image/celeba.py", line 125, in _split_generators
downloaded_dirs = dl_manager.download({
File "/home/felixt/.local/lib/python3.8/site-packages/tensorflow_datasets/core/download/download_manager.py", line 548, in download
return _map_promise(self._download, url_or_urls)
File "/home/felixt/.local/lib/python3.8/site-packages/tensorflow_datasets/core/download/download_manager.py", line 766, in _map_promise
res = tf.nest.map_structure(lambda p: p.get(), all_promises) # Wait promises
File "/home/felixt/.local/lib/python3.8/site-packages/tensorflow/python/util/nest.py", line 914, in map_structure
structure[0], [func(*x) for x in entries],
File "/home/felixt/.local/lib/python3.8/site-packages/tensorflow/python/util/nest.py", line 914, in
structure[0], [func(*x) for x in entries],
File "/home/felixt/.local/lib/python3.8/site-packages/tensorflow_datasets/core/download/download_manager.py", line 766, in
res = tf.nest.map_structure(lambda p: p.get(), all_promises) # Wait promises
File "/home/felixt/.local/lib/python3.8/site-packages/promise/promise.py", line 512, in get
return self._target_settled_value(_raise=True)
File "/home/felixt/.local/lib/python3.8/site-packages/promise/promise.py", line 516, in _target_settled_value
return self._target()._settled_value(_raise)
File "/home/felixt/.local/lib/python3.8/site-packages/promise/promise.py", line 226, in _settled_value
reraise(type(raise_val), raise_val, self._traceback)
File "/usr/lib/python3/dist-packages/six.py", line 703, in reraise
raise value
File "/home/felixt/.local/lib/python3.8/site-packages/promise/promise.py", line 844, in handle_future_result
resolve(future.result())
File "/usr/lib/python3.8/concurrent/futures/_base.py", line 437, in result
return self.__get_result()
File "/usr/lib/python3.8/concurrent/futures/_base.py", line 389, in __get_result
raise self._exception
File "/usr/lib/python3.8/concurrent/futures/thread.py", line 57, in run
result = self.fn(*self.args, **self.kwargs)
File "/home/felixt/.local/lib/python3.8/site-packages/tensorflow_datasets/core/download/downloader.py", line 217, in _sync_download
with _open_url(url, verify=verify) as (response, iter_content):
File "/usr/lib/python3.8/contextlib.py", line 113, in enter
return next(self.gen)
File "/home/felixt/.local/lib/python3.8/site-packages/tensorflow_datasets/core/download/downloader.py", line 279, in _open_with_requests
_assert_status(response)
File "/home/felixt/.local/lib/python3.8/site-packages/tensorflow_datasets/core/download/downloader.py", line 306, in _assert_status
raise DownloadError('Failed to get url {}. HTTP code: {}.'.format(
tensorflow_datasets.core.download.downloader.DownloadError: Failed to get url https://drive.google.com/uc?export=download&id=0B7EVK8r0v71pZjFTYXZWM3FlRnM&confirm=t. HTTP code: 404.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in tensorflow_datasets/image/celeba.py at _split_generators, which the traceback identifies as constructing the download URLs. Reproduce the provided download_and_prepare() call and inspect the failing Google Drive URL; done means the CelebA files download and preparation completes successfully.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, tensorflow
Domain
data, machine-learning
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.