The output dimensions of the hidden layer and what each dimension means
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 2.3k
- Forks
- 213
- PR merge metrics
- No merged PRs in 30d
Description
Hello, I am now studying the hidden state of the audio_encoder output of CLAP. I use hidden_states = encoder_output.hidden_states[-1], and after printing, I get torch.Size([1, 768, 8, 8]). What does each dimension mean here mean?Is it [batch_size, channels, time, freq]? Does that time represent frame? e.g. a piece of audio is divided into 8 frames, and then each frame is divided into the number of 8 frequency bins. In the end, the characteristic dimension of each frequency bin of each frame is 768?I'm very confused and hope you get an answer. Thank you very much
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at the audio_encoder entry point and inspect the CLAP implementation alongside the hidden_states output shape. Document what the batch, channel, time, and frequency axes represent for torch.Size([1, 768, 8, 8]), with an explanation that can be checked against the model output.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Documentation
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100