tensorflow / tensorflow/datasets

mock_data doesn't work with VOC when custom decoders are used

Open
#3,633 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
4.6k
Forks
1.6k
Avg merge
3h 54m
Merged PRs (30d)
1

Description

Short description
Attempting to iterate over the dataset returned by tfds.load('voc', decoders=...) when tfds.testing.mock_data is in effect results in an exception.

I assume this affects pretty much any dataset with a feature of type tfds.features.Sequence.

Environment information

  • Operating System: Ubuntu 18.04

  • Python version: 3.6.9

  • tensorflow-datasets/tfds-nightly version: tensorflow-datasets 4.4.0

  • tensorflow/tf-nightly version: tensorflow 2.6.2

  • Does the issue still exists with the last tfds-nightly package (pip install --upgrade tfds-nightly) ? Yes

Reproduction instructions

import tensorflow_datasets as tfds

with tfds.testing.mock_data():
    voc = tfds.load('voc', split='test', decoders={
        'image': tfds.decode.SkipDecoding(),
    })
    print(next(iter(voc)))

Link to logs
https://gist.github.com/IRDonch/7275316c5a1323c9f4a22d002b8391d9

Expected behavior
Valid examples should be returned.

Additional context
N/A

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by running the provided tfds.load and tfds.testing.mock_data reproduction with the VOC dataset, then trace how custom decoders interact with tfds.features.Sequence. Done means iteration returns valid examples without an exception, with coverage for the reported decoder configuration.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, tensorflow
Domain
data
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
50/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.