tensorflow / tensorflow/datasets

mock_data should create fake split info

Open
#3,631 0 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Python
Stars
4.6k
Forks
1.6k
Avg merge
3h 54m
Merged PRs (30d)
1

Description

Is your feature request related to a problem? Please describe.
I'm trying to write unit tests for my code that uses TFDS, and I would like to run those tests with fake data, so that the test suite doesn't have to download the real datasets. tfds.testing.mock_data is almost suitable for this purpose; however, the code in question needs to access split information for the datasets (i.e., tfds.builder('xxx').info.splits), and mock_data does not create this information, leaving the split dictionary blank. So it's not directly usable for my purposes.

Describe the solution you'd like
When mock_data is in effect, builders' split dictionaries should be populated with fake data that is consistent with the num_examples parameter. It would be sufficient if there was just a single train split with the correct number of examples.

Describe alternatives you've considered
I'm aware that MockPolicy.USE_FILES lets you load the real split info from the metadata files. However, this solution is inconvenient, since you need to add metadata files to your repository; and it's not really what I want - I want fake split info that's consistent with the fake data, not split info from the real dataset.

Currently, I've settled on mocking DatasetBuilder.__init__ myself on top of mock_data to populate the split dictionary. However, this requires me to call DatasetInfo.set_splits, which is described as a "private method", so it's not a fully satisfactory solution either.

Additional context
N/A

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing tfds.testing.mock_data and how DatasetBuilder and DatasetInfo currently represent splits, including the mentioned DatasetInfo.set_splits method. Add fake split metadata consistent with num_examples, with a train split, and verify that builders expose the expected split information while mock_data is active.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, tensorflow
Domain
testing-qa
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.