Firepal / Firepal/stammer

composition of audio match frames rather than selection of single match frames

Open
#23 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
504
Forks
38
Avg merge
9h 44m
Merged PRs (30d)
9

Description

If the audio match frames are a linear combination of the best matches, then the audio output could be tuned to be more or less like the source audio--truncation at one element is effectively the current implementation.
This also leaves open the possibility of interesting combinations of the visual frames (i.e, the visual frames corresponding to the audio matches [basis vectors]).
One straightforward visualization would be vertical slices with the width assigned as a fraction of the frame corresponding to its coefficient in the linear combination.

Another possibility is the use of a graphic equalizer-like effect--but which would only work well if there are time periods with relatively moderate changes to the basis vectors/frames. Each bar in the equalizer then represents a frame and the coefficient is its height. A bar could drop out as the influence of the particular frame fell to zero.
geq

Contributor guide

No contributing guide indexed for this repository

Research direction

No files, tests, or entry points are named. Start by locating the current single-match selection and truncation behavior, then investigate how linear combinations could tune the audio and how corresponding visual frames could be composed; done means the proposed combination and at least one visualization are implemented and demonstrably usable.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
audio-video-rtc, data-visualization
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.