New interface for MVDR beamforming
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 2.9k
- Forks
- 799
- Avg merge
- 58m
- Merged PRs (30d)
- 3
Description
🚀 The feature
It would be very helpful to provide the following interface for the beamforming module (torchaudio.transforms.MVDR):
forward(specgram: torch.Tensor, psd_s: torch.Tensor, psd_n: torch.Tensor) → torch.Tensor
and maybe add some high-level glue functions that takes the masks as input, but has only a few lines of code.
Motivation, pitch
The current forward method of torchaudio.transforms.MVDR only accepts spectrogram and masks as input, and calculates the PSD matrices internally.
The current design is easy to use mainly for mask-based beamforming, but it may lose the flexibility for the users to:
- use their own PSD estimates (e.g. mask-based PSDs, neural-network-based PSD estimates in ADL-MVDR and NICE-BEAM)
- adjust PSD estimates with some additional regularization/stabilization techniques (e.g. PSD diagonal loading and PSD eigenvalue flooring)
Because in different setups, a different stabilization may be ideal or preferred. It would be better to allow the user to make the choice.
- combine with conventional blind beamforming methods
- etc.
Alternatives
No response
Additional context
No response
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the torchaudio.transforms.MVDR.forward entry point and compare its current spectrogram-and-mask interface with the proposed specgram, psd_s, and psd_n tensors. Confirm that externally supplied PSD estimates and the existing mask-based workflow are both covered; any high-level mask helpers are optional in the request.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- audio-video-rtc
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100