tensorflow / tensorflow/datasets

`wmt_t2t_translate` fails build when `--data_dir` set to GCS location

Open
#4,680 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
4.6k
Forks
1.6k
Avg merge
3h 54m
Merged PRs (30d)
1

Description

/!\ PLEASE INCLUDE THE FULL STACKTRACE AND CODE SNIPPET

Short description
wmt_t2t_translate will not build when --data_dir is set to a GCS location due to etils.Path().glob() behavior when glob pattern is a gcs URI

Environment information

  • Operating System: Linux

  • Python version: 3.9.15

  • tensorflow-datasets/tfds-nightly version: 4.8.2 / 4.8.2.dev202301270045

  • tensorflow/tf-nightly version: 2.11.0

  • Does the issue still exists with the last tfds-nightly package (pip install --upgrade tfds-nightly) ?

  • yes

Reproduction instructions

  • pip install tensorflow tensorflow-datasets
  • tfds build --data_dir $GCS_DATA_DIR wmt_t2t_translate
INFO[wmt.py]: Using sub-datasets: {Split('train'): ['europarl_v7', 'commoncrawl', 'newscommentary_v13'], Split('validation'): ['newstest2013'], Split('test'): ['newstest2014']}
INFO[wmt.py]: Generating examples from: europarl_v7                                                                                                                                                                                                                                                                                                                                                                                   
Traceback (most recent call last):                                                                                                                                                                                                                                                                                                                                                                                                    
  File "/opt/conda/envs/tfds-bug/bin/tfds", line 8, in <module>                                                                                                                                                                                                                                                                                                                                                                       
    sys.exit(launch_cli())
  File "/opt/conda/envs/tfds-bug/lib/python3.9/site-packages/tensorflow_datasets/scripts/cli/main.py", line 109, in launch_cli
    app.run(main, flags_parser=_parse_flags)
  File "/opt/conda/envs/tfds-bug/lib/python3.9/site-packages/absl/app.py", line 308, in run
    _run_main(main, args)
  File "/opt/conda/envs/tfds-bug/lib/python3.9/site-packages/absl/app.py", line 254, in _run_main
    sys.exit(main(argv))
  File "/opt/conda/envs/tfds-bug/lib/python3.9/site-packages/tensorflow_datasets/scripts/cli/main.py", line 104, in main
    args.subparser_fn(args)
  File "/opt/conda/envs/tfds-bug/lib/python3.9/site-packages/tensorflow_datasets/scripts/cli/build.py", line 274, in _build_datasets
    _download_and_prepare(args, builder)
  File "/opt/conda/envs/tfds-bug/lib/python3.9/site-packages/tensorflow_datasets/scripts/cli/build.py", line 510, in _download_and_prepare
    builder.download_and_prepare(
  File "/opt/conda/envs/tfds-bug/lib/python3.9/site-packages/tensorflow_datasets/core/logging/__init__.py", line 169, in __call__
    return function(*args, **kwargs)
  File "/opt/conda/envs/tfds-bug/lib/python3.9/site-packages/tensorflow_datasets/core/dataset_builder.py", line 642, in download_and_prepare
    self._download_and_prepare(
  File "/opt/conda/envs/tfds-bug/lib/python3.9/site-packages/tensorflow_datasets/core/dataset_builder.py", line 1487, in _download_and_prepare
    future = split_builder.submit_split_generation(
  File "/opt/conda/envs/tfds-bug/lib/python3.9/site-packages/tensorflow_datasets/core/split_builder.py", line 340, in submit_split_generation
    return self._build_from_generator(**build_kwargs)
  File "/opt/conda/envs/tfds-bug/lib/python3.9/site-packages/tensorflow_datasets/core/split_builder.py", line 406, in _build_from_generator
    for key, example in utils.tqdm(
  File "/opt/conda/envs/tfds-bug/lib/python3.9/site-packages/tqdm/std.py", line 1195, in __iter__
    for obj in iterable:
  File "/opt/conda/envs/tfds-bug/lib/python3.9/site-packages/tensorflow_datasets/translate/wmt.py", line 938, in _generate_examples
    for sub_key, ex in sub_generator(*files):
  File "/opt/conda/envs/tfds-bug/lib/python3.9/site-packages/tensorflow_datasets/translate/wmt.py", line 1000, in _parse_parallel_sentences
    assert f1_files and f2_files, "No matching files found: %s, %s." % (f1, f2)
AssertionError: No matching files found: gs://gcp-ml-sandbox-t5x/datasets/downloads/extracted/TAR_GZ.statmt.org_wmt13_traini-parall-europa-v7AiTHxxDIoGPf2JOwzAgwIC1h9MdcF-uOMYNhA9J9luc.tgz/training/europarl-v7.de-en.de, gs://gcp-ml-sandbox-t5x/datasets/downloads/extracted/TAR_GZ.statmt.org_wmt13_traini-parall-europa-v7AiTHxxDIoGPf2JOwzAgwIC1h9MdcF-uOMYNhA9J9luc.tgz/training/europarl-v7.de-en.en.
(tfds-bug) jupyter@python3-conda-gpu:~/T5X$ 

If you share a colab, make sure to update the permissions to share it.

Link to logs
If applicable, <link to gist with logs, stack trace>

Expected behavior
dataset to be built

Additional context
Seems to be an issue with file globbing behavior being different for gcs vs local paths

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the tfds build --data_dir $GCS_DATA_DIR wmt_t2t_translate reproduction and read tensorflow_datasets/translate/wmt.py, especially _parse_parallel_sentences near the failing assertion. Compare how etils.Path().glob() handles the GCS paths shown in the traceback versus local paths. Done means the dataset builds successfully with a GCS --data_dir without the no-matching-files assertion.

Written by the indexing model from the issue text.

Assessment

Tech stack
google-cloud, python
Domain
cloud, data-engineering
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.