meta-pytorch / meta-pytorch/torchcodec
Add a sampler that samples only keyframes (I-Frames)
Open
Nobody has claimed this yet.
enhancement
- Dominant language
- Python
- Stars
- 1.2k
- Forks
- 125
- Avg merge
- 22h 47m
- Merged PRs (30d)
- 54
Description
Starting clips on keyframes can potentially be a fast and useful sampler because it can sometimes be faster to seek and decode to a keyframe instead of decoding multiple P frames to get to the sample point.
Ideas:
- There can be an API option to to return clips whose first frame is always a keyframe
- There can be an API option to have clips start on keyframes if they are "close enough" (in pts) to the sample point that the sampler wanted to choose. This can be used to balance performance and scene diversity
- This should work in approximate mode (i.e. it shouldn't require a full scan of the file to do this -- we could read the list of keyframes from the header)
- There could potentially be an API call that returns the list of pts to the user, if they want to do something with it (they could look at it and decide whether it's worth doing keyframe only sampling or uniform sampling)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The issue does not name files, tests, or an entry point. Begin by locating the existing sampler API and its approximate-mode behavior, then clarify which keyframe-sampling option should be implemented. Done means one agreed API behavior is implemented, including the required keyframe selection and approximate-mode support, with tests covering it.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- audio-video-rtc
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100