tensorflow / tensorflow/datasets
Slow 'encode_example' for Sequence
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 4.6k
- Forks
- 1.6k
- Avg merge
- 3h 54m
- Merged PRs (30d)
- 1
Description
Short description
Probably just like #1093 but for encoding. We have a lot of data stored in tfds.features.Sequence(tf.int64) and tfds.features.Sequence(tf.float32) and the more data we add the slower dataset creation becomes.
Some measurements:
Default dataset - 14min 8s
Dataset with sequence features values set to empty list - 16s
All sequence feature values wrapped in np.array - 12min 46s
With those lines commented https://github.com/tensorflow/datasets/blob/v1.3.0/tensorflow_datasets/core/features/sequence_feature.py#L142-L145 - 5min 32s (dataset seems to be correct after it)
Environment information
- Operating System: Debian GNU/Linux 10 (buster)
- Python version: 3.7.5
tensorflow-datasets/tfds-nightlyversion: tensorflow-datasets==1.3.0tensorflow/tensorflow-gpu/tf-nightly/tf-nightly-gpuversion: tensorflow==1.14.0
Reproduction instructions
Create a dataset with sequence of int/float features, populate each feature w/ a list of 10k values, measure dataset creation time. Comment out lines at https://github.com/tensorflow/datasets/blob/v1.3.0/tensorflow_datasets/core/features/sequence_feature.py#L142-L145 , re-create dataset, measure creation time, verify the dataset is the same.
Probably some existing dataset can expose the behaviour as well.
Link to logs
Code in question converts a list like this:
[0.20000000298023224, 0.4000000059604645, 0.0, 0.0, 0.20000000298023224, 0.10000000149011612, 0.44999998807907104, 0.05000000074505806, 0.0,
into
[array(0.2, dtype=float32), array(0.4, dtype=float32), array(0., dtype=float32), array(0., dtype=float32), array(0.2, dtype=float32), array(0.1, dtype=float32), array(0.45, dtype=float32), array(0.05, dtype=float32), array(0., dtype=float32),
Guess multiple array creation is the cause of slowness.
Expected behavior
Sequence of numbers shouldn't take too much time to be encoded.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in tensorflow_datasets/core/features/sequence_feature.py around lines 142-145, then reproduce the reported dataset creation case with Sequence features containing lists of 10k integers or floats. Compare creation times with those lines enabled or commented out, and verify that the resulting datasets remain identical while encoding no longer creates the reported slowdown.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- numpy, python
- Domain
- data, performance
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100