google-deepmind / google-deepmind/sonnet
Can you explain the logic of updated_ema_cluster_size in VectorQuantizerEMA?
Open
- Dominant language
- Python
- Stars
- 10k
- Forks
- 1.3k
- PR merge metrics
- No merged PRs in 30d
Description
Hi,
It seems `updated_ema_cluster_size` in `VectorQuantizerEMA` use small epsilon to avoid zero in denominator, but the formula is strange to me, so can you explain the logic here, and why not directly use something like `x=x+epsilon`?
https://github.com/deepmind/sonnet/blob/d1cd37117bcb98223b3e4b930717d418abb76484/sonnet/src/nets/vqvae.py#L270-L271
Contributor guide
Assessment
This issue has not been assessed yet.