lucidrains / lucidrains/vector-quantize-pytorch

LatentQuantize exploding loss

Open
#151 2 comments 1 reaction 0 assignees View on GitHub
Dominant language
Python
Stars
4k
Forks
338
PR merge metrics
No merged PRs in 30d

Description

I'm trying to use the LatentQuantize model in an autoencoder context. My inputs are flat 1-d tensors (32) and my encoder passes a shape of (batch_size, 64) to the quantizer. For now, my "levels" is [8, 6, 4], my latent_dim is 64:

```
self.lq = LatentQuantize(
levels=levels,
dim=latent_dim,
commitment_loss_weight=0.1,
quantization_loss_weight=0.1,
)
```

The loss starts at zero, then exponentially increases:
Screenshot 2024-08-01 at 1 53 43 PM

Any thoughts as to why this might happen?

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the LatentQuantize entry point and reproduce the reported autoencoder setup: flat 1-D inputs of 32, an encoder output of (batch_size, 64), levels [8, 6, 4], and latent_dim 64. Trace the loss from its initial zero value through the exponential increase, and document the conditions or configuration that produce the behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.