tensorflow / tensorflow/datasets

'huggingface:mnist' dataset not generating

Open
#3,148 9 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
4.6k
Forks
1.6k
Avg merge
3h 54m
Merged PRs (30d)
1

Description

Short description
Loading the huggingface:mnist gives ArributeError

Environment information

  • Operating System: Windows 10

  • Python version: 3.8

  • tensorflow-datasets/tfds-nightly version: 4.2.0+nightly

  • tensorflow/tf-nightly version: 2.4.0

  • Does the issue still exists with the last tfds-nightly package (pip install --upgrade tfds-nightly) ? Yes

Reproduction instructions

>>> nik = tfds.load('huggingface:mnist')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "f:\tfds\datasets\tensorflow_datasets\core\load.py", line 325, in load
    dbuilder = builder(name, data_dir=data_dir, try_gcs=try_gcs, **builder_kwargs)
  File "f:\tfds\datasets\tensorflow_datasets\core\load.py", line 172, in builder
    return cls(**builder_kwargs)  # pytype: disable=not-instantiable
  File "f:\tfds\datasets\tensorflow_datasets\core\dataset_builder.py", line 947, in __init__
    super().__init__(**kwargs)
  File "f:\tfds\datasets\tensorflow_datasets\core\dataset_builder.py", line 182, in __init__
    self.info.initialize_from_bucket()
  File "f:\tfds\datasets\tensorflow_datasets\core\utils\py_utils.py", line 149, in __get__
    cached = self.fget(obj)  # pytype: disable=attribute-error
  File "f:\tfds\datasets\tensorflow_datasets\core\community\huggingface_wrapper.py", line 124, in info
    info = self._info()
  File "C:\Users\nikhil\.cache\tensorflow_datasets\modules\tfds_community\huggingface\mnist\cf39692eaead13bb56fce5bcd2a1b230769d6801ce84feca3620e4201fdb8706\mnist.py", line 68, in _info
    "image": datasets.Array2D(shape=(28, 28), dtype="uint8"),
AttributeError: Failed to construct dataset huggingface:mnist: module 'datasets' has no attribute 'Array2D'

Expected behavior
Since, the datasets library has a well defined Array2D method, so there souldn't have been any error and the dataset should have been generated seamlessly.

Additional context
This issue might be part of a larger problem with interfacing huggingface datasets with TFDS as there seems to be unusual errors with other HF datasets in TFDS as well like huggingface:xnli etc.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with tensorflow_datasets/core/community/huggingface_wrapper.py and the generated huggingface:mnist module shown in the traceback, then reproduce the failure with tfds.load('huggingface:mnist'). Check how the installed datasets library exposes Array2D and related Hugging Face dataset types; done means the MNIST dataset loads without the reported AttributeError and the related compatibility behavior is understood.

Written by the indexing model from the issue text.

Assessment

Tech stack
huggingface, python
Domain
data
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.