Loading Data with DALI results in Protobuf String error
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 5.8k
- Forks
- 678
- Avg merge
- 3d 1h
- Merged PRs (30d)
- 27
Description
Tensorflow version: 2.4.1
Cuda version: 11
OS: Ubuntu 20.04
I've constructed a DALI pipeline that takes in tfrecords and is consumed as a tf.Dataset. When I run it with my code, I get the following error:
python3: ./google/protobuf/stubs/stringpiece.h:229: google::protobuf::StringPiece::StringPiece(const char*, google::protobuf::stringpiece_ssize_type): Assertion `len >= 0' failed.
I initially thought the error was in unpacking the jpeg data out of my tfrecords (stored as tf.io.FixedLenFeature([], tf.string)), but modifying my pipeline to not decode those fields doesn't fix the issue.
I've also tried fooling around with the data format (reducing the number of label types, etc.) but that doesn't seem to work.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Begin with the linked dali_loader.py and keras_predictor.py files, then reproduce the failure using TensorFlow 2.4.1, CUDA 11, and Ubuntu 20.04. Inspect the DALI pipeline and its tf.Dataset consumption; done means the protobuf StringPiece assertion is resolved or the incompatibility is identified with a documented reproduction.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, python
- Domain
- data-engineering, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100