lucidrains / lucidrains/vector-quantize-pytorch
Missing parameter of beta
- Dominant language
- Python
- Stars
- 4k
- Forks
- 338
- PR merge metrics
- No merged PRs in 30d
Description
Hi, in the original VQVAE paper, the commit_loss is defined as
```
(quantize.detach()-x) ** 2 + beta * (quantize - x.detach() ** 2)
```
where the beta is usually to be 0.25. But the commit_loss is defined as the following in your implementation:
```
F.mse_loss(quantize.detach(), x)
```
So I wonder if the parameter beta is set to be 1 by default or if the second term is missing? Thank you very much.
Contributor guide
No contributing guide indexed for this repository
Research direction
No file or test is named. Start by locating the commit_loss implementation and its use of F.mse_loss, then compare it with the VQVAE equation and the stated beta value. Done means the implementation's beta behavior and the possible missing term are resolved and verified against the intended formulation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100