google-deepmind / google-deepmind/sonnet

I have a question about the perplexity term (in the VQ-VAE).

Open
#257 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
10k
Forks
1.3k
PR merge metrics
No merged PRs in 30d

Description

As far as I understood, the perplexity used in this repo's VQ-VAE is kind of "meaningfully used codebook token numbers".

When only one codebook token is used, perplexity is 1.
When all codebook tokens appear uniformly, the perplexity equals the codebook nums.

So I was wondering, for good output quality, what is the minimum threshold of "perplexity divided by codebook nums"?
(I guess this should be found experimentally. If you have any results related to this question, it would be great to know.)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.