tensorflow / tensorflow/datasets
tensorflow_datasets v4.9.4 introduces bug that prevents loading datasets
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 4.6k
- Forks
- 1.6k
- Avg merge
- 3h 54m
- Merged PRs (30d)
- 1
Description
Short description
When upgrading to the most recent tensorflow_datasets==4.9.4 I am getting errors for loading datasets (from the official TFDS catalogue). I have verified that the same datasets can load in version 4.9.3 without problem.
Environment information
-
Operating System: verified on Colab -- https://colab.research.google.com/drive/1neCJ3_TnF1tqr8qv4FM5__-v4SwVOOxJ?usp=sharing
-
tensorflow-datasets/tfds-nightlyversion: 4.9.4 -
Does the issue still exists with the last
tfds-nightlypackage (pip install --upgrade tfds-nightly) ? -- Yes! (see Colab)
Reproduction instructions
import tensorflow_datasets as tfds
ds = tfds.load("fractal20220817_data", data_dir="gs://gresearch/robotics")
OR colab: https://colab.research.google.com/drive/1neCJ3_TnF1tqr8qv4FM5__-v4SwVOOxJ?usp=sharing
Link to logs
FileNotFoundError Traceback (most recent call last)
[/usr/local/lib/python3.10/dist-packages/tensorflow_datasets/core/dataset_info.py](https://localhost:8080/#) in read_from_json(path)
1033 try:
-> 1034 json_str = epath.Path(path).read_text()
1035 except OSError as e:
27 frames
FileNotFoundError: [Errno 2] No such file or directory: 'fractal20220817_data/0.1.0/dataset_info.json'
The above exception was the direct cause of the following exception:
FileNotFoundError Traceback (most recent call last)
FileNotFoundError: Could not load dataset info from fractal20220817_data/0.1.0/dataset_info.json
The above exception was the direct cause of the following exception:
FileNotFoundError Traceback (most recent call last)
[/usr/local/lib/python3.10/dist-packages/tensorflow_datasets/core/utils/py_utils.py](https://localhost:8080/#) in reraise(e, prefix, suffix)
383 else:
384 exception = RuntimeError(f'{type(e).__name__}: {msg}')
--> 385 raise exception from e
386 # Otherwise, modify the exception in-place
387 elif len(e.args) <= 1:
FileNotFoundError: Failed to construct dataset "fractal20220817_data", builder_kwargs "{'data_dir': 'gs://gresearch/robotics'}": Could not load dataset info from fractal20220817_data/0.1.0/dataset_info.json
Additional context
Interestingly, constructing a builder_from_directory still seems to work even in the most recent tfds version.
builder = tfds.builder_from_directory("gs://gresearch/robotics/fractal20220817_data/0.1.0")
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the Colab reproduction using tfds.load("fractal20220817_data", data_dir="gs://gresearch/robotics") and compare behavior between tensorflow-datasets 4.9.3 and 4.9.4. Trace the dataset-info loading path and compare it with builder_from_directory, which still works; done means the official dataset loads successfully from the documented data directory.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, tensorflow
- Domain
- data
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100