tensorflow / tensorflow/datasets
Corrupt JPEG data: 211 extraneous bytes before marker 0xd9
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 4.6k
- Forks
- 1.6k
- Avg merge
- 3h 54m
- Merged PRs (30d)
- 1
Description
What I need help with / What I was wondering
I have a training task. And my data is a bunch of videos. When I use tfds as dataload, it will generate tf record for me first, but when I train, the following error will appear.
What I've tried so far
To solve this problem, I tried to use dataset = dataset.ignore_errors() to skip these problematic videos, but this method does not seem to work
It would be nice if...
Environment information
ubuntu
Python 3.10.12
tensorflow 2.17.0
tensorflow-datasets 4.9.6
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No repository file, test, or entry point is named. Start by reproducing the TFDS-to-training pipeline with the reported Ubuntu, Python, TensorFlow, and TFDS versions, then identify where the JPEG warning occurs and whether ignore_errors is applied. Done would require a confirmed scope and a reproducible way to handle the problematic video records.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, tensorflow
- Domain
- data, machine-learning
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100