developmentseed / developmentseed/bioacoustics-api
Filtering for speech and empty
- Dominant language
- Jupyter Notebook
- Stars
- 1
- Forks
- 0
- PR merge metrics
- No merged PRs in 30d
Description
Hi, ya'll!
Seems like we missed a step in filtering the speech data. I added logits for speech and empty classes to the TFRecord embeddings, but then we need to use them to ignore/drop certain segments.
When indexing the embeddings, we should omit:
* The full embedding if any channel has a speech logit above -0.25,
* The ‘raw audio’ (0th) channel if the ‘empty’ logit is above 1.5,
* Separated channels (1-4) if the ‘empty’ logit is above 0.0.
The problem is that the embeddings are normally in a large-ish block, so we can't drop single embeddings from the set easily. Instead, I've included the logits so that the needed logic can be applied downstream.
Alternatively, I could either a) add some 'to_filter' labeled output matching the embedding batch dimensions ([T, C]), _or_ replace some embeddings with all-zeros, and then we can drop any all-zero embeddings during indexing.
I think all options require at least some additional downstream embedding, though.
WDYT?
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.