tensorflow / tensorflow/datasets
'huggingface:mnist' dataset not generating
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 4.6k
- Forks
- 1.6k
- Avg merge
- 3h 54m
- Merged PRs (30d)
- 1
Description
Short description
Loading the huggingface:mnist gives ArributeError
Environment information
-
Operating System: Windows 10
-
Python version: 3.8
-
tensorflow-datasets/tfds-nightlyversion: 4.2.0+nightly -
tensorflow/tf-nightlyversion: 2.4.0 -
Does the issue still exists with the last
tfds-nightlypackage (pip install --upgrade tfds-nightly) ? Yes
Reproduction instructions
>>> nik = tfds.load('huggingface:mnist')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "f:\tfds\datasets\tensorflow_datasets\core\load.py", line 325, in load
dbuilder = builder(name, data_dir=data_dir, try_gcs=try_gcs, **builder_kwargs)
File "f:\tfds\datasets\tensorflow_datasets\core\load.py", line 172, in builder
return cls(**builder_kwargs) # pytype: disable=not-instantiable
File "f:\tfds\datasets\tensorflow_datasets\core\dataset_builder.py", line 947, in __init__
super().__init__(**kwargs)
File "f:\tfds\datasets\tensorflow_datasets\core\dataset_builder.py", line 182, in __init__
self.info.initialize_from_bucket()
File "f:\tfds\datasets\tensorflow_datasets\core\utils\py_utils.py", line 149, in __get__
cached = self.fget(obj) # pytype: disable=attribute-error
File "f:\tfds\datasets\tensorflow_datasets\core\community\huggingface_wrapper.py", line 124, in info
info = self._info()
File "C:\Users\nikhil\.cache\tensorflow_datasets\modules\tfds_community\huggingface\mnist\cf39692eaead13bb56fce5bcd2a1b230769d6801ce84feca3620e4201fdb8706\mnist.py", line 68, in _info
"image": datasets.Array2D(shape=(28, 28), dtype="uint8"),
AttributeError: Failed to construct dataset huggingface:mnist: module 'datasets' has no attribute 'Array2D'
Expected behavior
Since, the datasets library has a well defined Array2D method, so there souldn't have been any error and the dataset should have been generated seamlessly.
Additional context
This issue might be part of a larger problem with interfacing huggingface datasets with TFDS as there seems to be unusual errors with other HF datasets in TFDS as well like huggingface:xnli etc.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with tensorflow_datasets/core/community/huggingface_wrapper.py and the generated huggingface:mnist module shown in the traceback, then reproduce the failure with tfds.load('huggingface:mnist'). Check how the installed datasets library exposes Array2D and related Hugging Face dataset types; done means the MNIST dataset loads without the reported AttributeError and the related compatibility behavior is understood.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- huggingface, python
- Domain
- data
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100