composition of audio match frames rather than selection of single match frames
- Dominant language
- Python
- Stars
- 504
- Forks
- 38
- Avg merge
- 9h 44m
- Merged PRs (30d)
- 9
Description
If the audio match frames are a linear combination of the best matches, then the audio output could be tuned to be more or less like the source audio--truncation at one element is effectively the current implementation.
This also leaves open the possibility of interesting combinations of the visual frames (i.e, the visual frames corresponding to the audio matches [basis vectors]).
One straightforward visualization would be vertical slices with the width assigned as a fraction of the frame corresponding to its coefficient in the linear combination.
Another possibility is the use of a graphic equalizer-like effect--but which would only work well if there are time periods with relatively moderate changes to the basis vectors/frames. Each bar in the equalizer then represents a frame and the coefficient is its height. A bar could drop out as the influence of the particular frame fell to zero.
Contributor guide
No contributing guide indexed for this repository
Research direction
No files, tests, or entry points are named. Start by locating the current single-match selection and truncation behavior, then investigate how linear combinations could tune the audio and how corresponding visual frames could be composed; done means the proposed combination and at least one visualization are implemented and demonstrably usable.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- audio-video-rtc, data-visualization
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100