`F.vad` batch consistency behaviour
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 2.9k
- Forks
- 799
- Avg merge
- 58m
- Merged PRs (30d)
- 3
Description
I isolated a potential issue with batch consistency and F.vad in #1341:
[Describing the change] Tweak the VAD consistency tests by using longer samples for the random test, and using lower thresholds for the file test. This now causes many of the VAD tests to fail.
This has me thinking - if the purpose of
F.vadis to outright crop the input tensor (not, for example, zero it), shouldn't we expect it to produce different results on a sample vs. a batch? I believe my tweaks just contain the right conditions for that to happen, but I might be misreading. If this is the case, shouldF.vadbe removed from the batch consistency tests?
@mthrok replied:
I see. I did not review the design of
F.vad/T.Vadin detail, but your finding sounds like that the design ofT.Vadis ill-formed. (I thought thatF.vadwould zero-out the non-vocal part). To me it does not make sense to batch the results that could be cropped in any way. We should check the behavior of thesoxcommand, but @astaff do you have an idea of how it is supposed to behave?
Opening a dedicated issue so the discussion can continue here.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the F.vad and T.Vad behavior and the VAD consistency tests discussed in this issue. Check the behavior of the sox command, then determine and test the expected relationship between sample and batch results, including whether cropping is compatible with batch consistency.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- audio-video-rtc
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100