huggingface / huggingface/diffusers
Combined loss term for VQ-VAE (`diffusers.VQModel`)
- 主要言語
- Python
- スター
- 34.5k
- フォーク
- 7.3k
- 平均マージ
- 3日 3時間
- マージ済み PR(30日)
- 91
説明
For training the VQ-VAE component of a latent diffusion model a la `CompVis/ldm-celebahq-256` (which uses `diffusers.VQModel`), is there a combined loss term for each of the losses as described by the authors: reconstruction loss, vq loss, and commitment loss?
I see the vq loss term is collected in `VectorQuantizer`, but it does not seem to be used anywhere else.
https://github.com/huggingface/diffusers/blob/ebc99a77aad647c5d33eb36a33c23f7b3949cb40/src/diffusers/models/autoencoders/vae.py#L726-L730
I'm also open to alternatives to `VQModel` like `AutoEncoderKL`, if they can collect the loss terms more easily.
Thank you!
コントリビューションガイド
調査の方向性
Start with the linked lines in src/diffusers/models/autoencoders/vae.py and trace how VectorQuantizer's VQ loss is handled by VQModel. Determine whether reconstruction, VQ, and commitment losses can be collected together, and verify that the resulting loss terms are available for the stated VQ-VAE training use case.
索引モデルが issue の本文から書いたものです。
評価
- 技術スタック
- python, pytorch
- 領域
- machine-learning
- issue の種類
- 機能追加
- 難易度
- 4/5
- 見積もり時間
- 3〜5日
- 活発さ
- 停滞
- 明瞭さ
- 説明が足りない
- 初心者へのやさしさ
- 30/100