lucidrains / lucidrains/vector-quantize-pytorch
Some clarification required
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 4k
- Forks
- 338
- PR merge metrics
- No merged PRs in 30d
Description
Hi Phil,
I was experimenting with FSQ/LFQ for 3d-motion autoencoder and was wondering how to work through LFQ variety of options?
With FSQ its quite straight forward as it does not have any losses and you just need to find suitable levels for particular task.
However when I start training LFQ immediately I face negative aux_loss issue which is of course due to entropy_aux_loss.
I understand that we can combat it with decreasing diversity but i dont like the idea, another approach would be lowering entropy_loss_weight which can lead to poor codebook utilisation i believe...
So what are the options to tune LFQ parameters? btw even with negative aux_loss it still seems to train fine, at least reconstruction loss goes down
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The issue does not identify a file, test, or entry point. Start by locating the LFQ implementation and its entropy_aux_loss, diversity, and entropy_loss_weight parameters, then document the available tuning options and explain how to interpret negative aux_loss during training.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Documentation
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100