tensorflow / tensorflow/text

Dataset for tutorial missing

Open
#1,497 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
1.3k
Forks
379
Avg merge
3h 30m
Merged PRs (30d)
8

Description

The following tutorial and guide use a dataset that is missing:

Neural machine translation with a Transformer and Keras
https://www.tensorflow.org/text/tutorials/transformer

Subword tokenizers
https://www.tensorflow.org/text/guide/subwords_tokenizer

Both use the following line to load the dataset:
examples, metadata = tfds.load('ted_hrlr_translate/pt_to_en', with_info=True, as_supervised=True)
which attempts to download a file from
http://www.phontron.com/data/qi18naacl-dataset.tar.gz

This page also refers to the file qi18naacl-dataset.tar.gz
https://github.com/neulab/word-embeddings-for-nmt
leading to the same broken link.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by checking the Transformer tutorial and Subword tokenizers guide, along with their tfds.load('ted_hrlr_translate/pt_to_en') references. Confirm whether the dataset source is unavailable and identify the appropriate project-owned documentation or data change. Done means both guides no longer depend on a broken download and their dataset-loading examples work.

Written by the indexing model from the issue text.

Assessment

Domain
documentation
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.