[fyi][video] Video clip creation fails on YFCC
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 17.9k
- Forks
- 7.3k
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 13
Description
Culprit:
known bug in H264 decoding
For certain filetypes - specifically some related to YFCC100m dataset - there is an error whilst creating the VideoClips object. Not 100% sure what is the best way of fixing the issue (other than try-catch in read_video_timestamps.
Even if the fix is not critical, we should make sure that is caught in C++ or future versions.
Sample YFCC video in question:
27b66931edb88a5f7e6c756ea65fd8.mp4
there are at least 50 of them, probably more.
Minimal repro example
video_list = ['~/yfcc100m/mp4/27b/669/27b66931edb88a5f7e6c756ea65fd8.mp4']
VideoClips(video_list, 8, 1)
or even simpler
import av
container = av.open('~/yfcc100m/mp4/27b/669/27b66931edb88a5f7e6c756ea65fd8.mp4')
for idx, frame in enumerate(container.decode({'video': 0})):
pass
Output error stack:
Invalid NAL unit size (34173 > 22144).
Error splitting the input into NAL units.
stream 1, offset 0x16baf66: partial file
Invalid NAL unit size (24747 > 6180).
Error splitting the input into NAL units.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the VideoClips construction path and read_video_timestamps, then reproduce the failure using 27b66931edb88a5f7e6c756ea65fd8.mp4 and the av.open decoding example from the issue. Determine how the H264 decoding error should be handled, and verify that creating VideoClips no longer fails unexpectedly for the affected YFCC files.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- computer-vision
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100