pytorch / pytorch/audio

Confusing IID behavior in masking transforms.

Open
#1,504 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
2.9k
Forks
799
Avg merge
58m
Merged PRs (30d)
3

Description

T.TimeMasking and T. FrequencyMasking have a parameter for changing the way mask is applied IID or not.

https://github.com/pytorch/audio/blob/1f136671b84071a2fe1d5b762df64f3a76310c31/torchaudio/transforms.py#L890-L899

https://github.com/pytorch/audio/blob/1f136671b84071a2fe1d5b762df64f3a76310c31/torchaudio/transforms.py#L905-L914

However this behavior is not automatically enforced, users have to make sure that the input Tensor is 4D by themselves manually, because of if ~ else clause here.

https://github.com/pytorch/audio/blob/1f136671b84071a2fe1d5b762df64f3a76310c31/torchaudio/transforms.py#L884-L887

This logic is error prune and can introduce subtle error, which is hard to detect. Instead it should be throwing an error if the input Tensor is not 4D.

        if self.iid_masks:
            if specgram.dim() != 4:
                throw ValueError(...)
            return F.mask_along_axis_iid(specgram, self.mask_param, mask_value, self.axis + 1)
        return F.mask_along_axis(specgram, self.mask_param, mask_value, self.axis)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in torchaudio/transforms.py at the TimeMasking and FrequencyMasking implementations and the linked conditional around lines 884-887. Check how iid_masks selects the masking function and how tensor dimensionality is handled. Done means invalid input dimensions for IID masking are rejected explicitly rather than silently taking the other path.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.