tensorflow / tensorflow/datasets
tfds.load downloads nsynth with missing parts
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 4.6k
- Forks
- 1.6k
- Avg merge
- 3h 54m
- Merged PRs (30d)
- 1
Description
Short description
I am using tfds.load to download nsynth dataset. But no matter how many times I try, It's only downloaded up until nsynth-train.tfrecord-00975-of-01024. It seems like 976 to 1023 are missing.
Environment information
- Operating System: Windows 10
- Python version: 3.7.3
tensorflow-datasets/tfds-nightlyversion: tensorflow-datasets@2.1.0tensorflow/tensorflow-gpu/tf-nightly/tf-nightly-gpuversion: tensorflow@2.1.0
Reproduction instructions
datasets = tfds.load("nsynth", data_dir="data")
Link to logs
Dl Completed...: 100%|██████████████████████████████████████████████████████████| 989/989 [2:28:10<00:00, 1.89s/ file]
Dl Completed...: 100%|██████████████████████████████████████████████████████████| 989/989 [2:28:10<00:00, 8.99s/ file]
Dataset nsynth downloaded and prepared to data\nsynth\full\2.3.0. Subsequent calls will reuse this data.
Expected behavior
I expect the whole of the dataset to be downloaded.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the issue with the shown tfds.load("nsynth", data_dir="data") call and compare the download log with the expected 1,024 shards. Investigate why the completed download reports only 989 files; done means the full NSynth dataset, including shards 976 through 1023, is downloaded and prepared.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data, machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100