lucidrains / lucidrains/vector-quantize-pytorch
No way of training the codebook
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 4k
- Forks
- 338
- PR merge metrics
- No merged PRs in 30d
Description
Hi!
Could you please explain how the codebook vectors are updated if the codebook vectors are not required to be orthogonal?
embedtensors in both Euclidean and CosineSim codebooks are registered as buffers, so they can't be updated at all- There is no loss on the codebook vectors that moves them closer to the input
Am I missing something? It seems that right now there is no way of updating the codebook vectors without the orthogonal loss.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating the Euclidean and CosineSim codebook implementations, then trace how their embed tensors are registered and how codebook-related losses are applied. Verify whether any update path exists without the orthogonal loss; done means the intended training behavior is implemented or clearly documented and covered by a regression test.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100