meta-pytorch / meta-pytorch/torchcodec
Live stream decoding
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.2k
- Forks
- 125
- Avg merge
- 22h 47m
- Merged PRs (30d)
- 54
Description
🚀 The feature
Currently, both audio and video decoding depend on knowing both the total number of frames and the duration of the stream. As a consequence, we can't decode live streaming video or audio because we cannot know the total number of frames or the duration.
This feature would be to somehow enable decoding of live media. Because the APIs for VideoDecoder and AudioDecoder assume the ability to seek to arbitrary places in the stream, I suspect we might need a new top-level public API specifically for live streams.
Motivation, pitch
On the training side, I suspect this feature will not be used much. That is, training tends to happen on large corpuses of pre-existing media. I doubt there's much demand to train on a live stream. But doing inference on a live stream seems like it would be a common use-case, and we can't currently support it.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the existing VideoDecoder and AudioDecoder APIs and how they depend on total frame counts, duration, and seeking. Define the public API needed for live streams and verify that live audio and video can be decoded without those known values.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- audio-video-rtc
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100