developmentseed / developmentseed/bioacoustics-api

Filtering for speech and empty

Open
#48 4 comments 0 reactions 1 assignee Claimed by @leothomas View on GitHub
Dominant language
Jupyter Notebook
Stars
1
Forks
0
PR merge metrics
No merged PRs in 30d

Description

Hi, ya'll!

Seems like we missed a step in filtering the speech data. I added logits for speech and empty classes to the TFRecord embeddings, but then we need to use them to ignore/drop certain segments.

When indexing the embeddings, we should omit:
* The full embedding if any channel has a speech logit above -0.25,
* The ‘raw audio’ (0th) channel if the ‘empty’ logit is above 1.5,
* Separated channels (1-4) if the ‘empty’ logit is above 0.0.

The problem is that the embeddings are normally in a large-ish block, so we can't drop single embeddings from the set easily. Instead, I've included the logits so that the needed logic can be applied downstream.

Alternatively, I could either a) add some 'to_filter' labeled output matching the embedding batch dimensions ([T, C]), _or_ replace some embeddings with all-zeros, and then we can drop any all-zero embeddings during indexing.

I think all options require at least some additional downstream embedding, though.

WDYT?

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.