tensorflow / tensorflow/datasets
gs' not implemented
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 4.6k
- Forks
- 1.6k
- Avg merge
- 3h 54m
- Merged PRs (30d)
- 1
Description
/!\ PLEASE INCLUDE THE FULL STACKTRACE AND CODE SNIPPET
Short description
Description of the bug.
Environment information
- Operating System: windows
- Python version: 3.11.8
tensorflow-datasetsversion: 4.9.4tensorflowversion: 2.15.0
Reproduction instructions
git clone https://github.com/tensorflow/datasets.git
cd datasets
pip install -e .
python -c "import tensorflow_datasets as tfds; tfds = tfds.load('berkeley_autolab_ur5')"
Link to logs
Traceback (most recent call last):
File "C:\Users\Zelong\Desktop\Berkeley data\datasets\tensorflow_datasets\core\utils\py_utils.py", line 420, in try_reraise
yield
File "C:\Users\Zelong\Desktop\Berkeley data\datasets\tensorflow_datasets\core\load.py", line 222, in builder
return cls(**builder_kwargs) # pytype: disable=not-instantiable
^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Zelong\Desktop\Berkeley data\datasets\tensorflow_datasets\core\logging\__init__.py", line 288, in decorator
return function(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Zelong\Desktop\Berkeley data\datasets\tensorflow_datasets\core\dataset_builder.py", line 1362, in __init__
super().__init__(**kwargs)
File "C:\Users\Zelong\Desktop\Berkeley data\datasets\tensorflow_datasets\core\logging\__init__.py", line 288, in decorator
return function(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Zelong\Desktop\Berkeley data\datasets\tensorflow_datasets\core\dataset_builder.py", line 285, in __init__
self.info.initialize_from_bucket()
^^^^^^^^^
File "C:\Users\Zelong\Desktop\Berkeley data\datasets\tensorflow_datasets\core\logging\__init__.py", line 168, in __call__
return function(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Zelong\Desktop\Berkeley data\datasets\tensorflow_datasets\core\dataset_builder.py", line 474, in info
info = self._info()
^^^^^^^^^^^^
File "C:\Users\Zelong\Desktop\Berkeley data\datasets\tensorflow_datasets\robotics\dataset_importer_builder.py", line 82, in _info
features = self.get_ds_builder().info.features
^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Zelong\Desktop\Berkeley data\datasets\tensorflow_datasets\robotics\dataset_importer_builder.py", line 150, in get_ds_builder
ds_builder = tfds.builder_from_directory(ds_location)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Zelong\Desktop\Berkeley data\datasets\tensorflow_datasets\core\read_only_builder.py", line 149, in builder_from_directory
return ReadOnlyBuilder(builder_dir=builder_dir)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Zelong\Desktop\Berkeley data\datasets\tensorflow_datasets\core\logging\__init__.py", line 288, in decorator
return function(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Zelong\Desktop\Berkeley data\datasets\tensorflow_datasets\core\read_only_builder.py", line 65, in __init__
info_proto = dataset_info.read_proto_from_builder_dir(builder_dir)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Zelong\Desktop\Berkeley data\datasets\tensorflow_datasets\core\dataset_info.py", line 1059, in read_proto_from_builder_dir
return read_from_json(info_path)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Zelong\Desktop\Berkeley data\datasets\tensorflow_datasets\core\dataset_info.py", line 1035, in read_from_json
json_str = epath.Path(path).read_text()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Zelong\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\site-packages\etils\epath\abstract_path.py", line 157, in read_text
return f.read()
^^^^^^^^
File "C:\Users\Zelong\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\site-packages\tensorflow\python\lib\io\file_io.py", line 116, in read
self._preread_check()
File "C:\Users\Zelong\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\site-packages\tensorflow\python\lib\io\file_io.py", line 77, in _preread_check
self._read_buf = _pywrap_file_io.BufferedInputStream(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
tensorflow.python.framework.errors_impl.UnimplementedError: File system scheme 'gs' not implemented (file: 'gs://gresearch/robotics/berkeley_autolab_ur5/0.1.0/dataset_info.json')
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "C:\Users\Zelong\Desktop\Berkeley data\datasets\tensorflow_datasets\core\logging\__init__.py", line 168, in __call__
return function(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Zelong\Desktop\Berkeley data\datasets\tensorflow_datasets\core\load.py", line 643, in load
dbuilder = _fetch_builder(
^^^^^^^^^^^^^^^
File "C:\Users\Zelong\Desktop\Berkeley data\datasets\tensorflow_datasets\core\load.py", line 498, in _fetch_builder
return builder(name, data_dir=data_dir, try_gcs=try_gcs, **builder_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.2288.0_x64__qbz5n2kfra8p0\Lib\contextlib.py", line 81, in inner
return func(*args, **kwds)
^^^^^^^^^^^^^^^^^^^
File "C:\Users\Zelong\Desktop\Berkeley data\datasets\tensorflow_datasets\core\logging\__init__.py", line 168, in __call__
return function(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Zelong\Desktop\Berkeley data\datasets\tensorflow_datasets\core\load.py", line 219, in builder
with py_utils.try_reraise(
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.2288.0_x64__qbz5n2kfra8p0\Lib\contextlib.py", line 158, in __exit__
self.gen.throw(typ, value, traceback)
File "C:\Users\Zelong\Desktop\Berkeley data\datasets\tensorflow_datasets\core\utils\py_utils.py", line 422, in try_reraise
reraise(e, *args, **kwargs)
File "C:\Users\Zelong\Desktop\Berkeley data\datasets\tensorflow_datasets\core\utils\py_utils.py", line 389, in reraise
raise exception from e
RuntimeError: UnimplementedError: Failed to construct dataset "berkeley_autolab_ur5", builder_kwargs "{'data_dir': None}": File system scheme 'gs' not implemented (file: 'gs://gresearch/robotics/berkeley_autolab_ur5/0.1.0/dataset_info.json')
Expected behavior
It should find and download the dataset
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the reported command for berkeley_autolab_ur5 and trace the failure through tensorflow_datasets/core/load.py, robotics/dataset_importer_builder.py, core/read_only_builder.py, and core/dataset_info.py. Compare the gs:// path handling with the TensorFlow file I/O call shown in the stack trace. Done means the dataset can be found and downloaded from the reported Windows environment without the gs filesystem error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, tensorflow
- Domain
- data, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100