tensorflow / tensorflow/datasets

mocking.mock_data broken by https://github.com/tensorflow/datasets/commit/e8d8966e888667091875fd885ab0803a6dc5a383

Open
#3,851 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
4.6k
Forks
1.6k
Avg merge
3h 54m
Merged PRs (30d)
1

Description

/!\ PLEASE INCLUDE THE FULL STACKTRACE AND CODE SNIPPET

Short description
https://github.com/tensorflow/datasets/commit/e8d8966e888667091875fd885ab0803a6dc5a383 broken mock_data().

Environment information

  • Operating System: Linux

  • Python version: 3.7.7

  • tensorflow-datasets/tfds-nightly version: 4.5.2.dev202203220044

  • Does the issue still exists with the last tfds-nightly package (pip install --upgrade tfds-nightly) ?

Reproduction instructions

  with mock_data(num_examples=40):
    builder = tfds.builder("imagenet2012", "...")
    ... 

Link to logs

input_tfds.py:74: in _build_dataset
    builder = tfds.builder(cfg.dataset_name, data_dir=cfg.data_dir)
/miniconda/envs/py377/lib/python3.7/site-packages/tensorflow_datasets/core/load.py:149: in builder
    community.community_register.has_namespace(name.namespace)):
/miniconda/envs/py377/lib/python3.7/site-packages/tensorflow_datasets/core/community/registry.py:124: in has_namespace
    return namespace in self.registers_per_namespace
/miniconda/envs/py377/lib/python3.7/site-packages/tensorflow_datasets/core/community/registry.py:121: in registers_per_namespace
    return self.namespace_config.registers_per_namespace()
/miniconda/envs/py377/lib/python3.7/site-packages/tensorflow_datasets/core/community/registry.py:93: in registers_per_namespace
    config = toml.loads(self.config_path.read_text())
/miniconda/envs/py377/lib/python3.7/site-packages/etils/epath/abstract_path.py:141: in read_text
    return f.read()
/miniconda/envs/py377/lib/python3.7/site-packages/tensorflow/python/lib/io/file_io.py:114: in read
    self._preread_check()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
 self = <tensorflow.python.platform.gfile.GFile object at 0x7f4e18542450>
     def _preread_check(self):
      if not self._read_buf:
        if not self._read_check_passed:
          raise errors.PermissionDeniedError(None, None,
                                             "File isn't open for reading")
        self._read_buf = _pywrap_file_io.BufferedInputStream(
>           compat.path_to_str(self.__name), 1024 * 512)
E       tensorflow.python.framework.errors_impl.NotFoundError: /miniconda/envs/py377/lib/python3.7/site-packages/tensorflow_datasets/community-datasets.toml; No such file or directory
 /miniconda/envs/py377/lib/python3.7/site-packages/tensorflow/python/lib/io/file_io.py:77: NotFoundError

Expected behavior
tfds does not try to read community-datasets.toml.

Additional context
Add any other context about the problem here.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the failure with mocking.mock_data and tfds.builder using the supplied Python snippet, then inspect the community registry path reached through has_namespace and registers_per_namespace. Compare that behavior with commit e8d8966e888667091875fd885ab0803a6dc5a383; done means mock_data no longer attempts to read the missing community-datasets.toml file.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data, machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.