lucidrains / lucidrains/vector-quantize-pytorch

Missing parameter of beta

Open
#22 3 comments 1 reaction 0 assignees View on GitHub
Dominant language
Python
Stars
4k
Forks
338
PR merge metrics
No merged PRs in 30d

Description

Hi, in the original VQVAE paper, the commit_loss is defined as

```
(quantize.detach()-x) ** 2 + beta * (quantize - x.detach() ** 2)
```

where the beta is usually to be 0.25. But the commit_loss is defined as the following in your implementation:

```
F.mse_loss(quantize.detach(), x)
```

So I wonder if the parameter beta is set to be 1 by default or if the second term is missing? Thank you very much.

Contributor guide

No contributing guide indexed for this repository

Research direction

No file or test is named. Start by locating the commit_loss implementation and its use of F.mse_loss, then compare it with the VQVAE equation and the stated beta value. Done means the implementation's beta behavior and the possible missing term are resolved and verified against the intended formulation.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.