microsoft / microsoft/TRELLIS

Sparse structureVAE train

Open
#218 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
13.7k
Forks
1.3k
PR merge metrics
No merged PRs in 30d

Description

When training Sparse structureVAE,
TRELLIS/TRELLIS/trainers/vae/sparse_structure_vae py
z, mean, logvar = self.training_models['encoder'](ss.float(), sample_posterior=True, return_raw=True)
True is used, which is a common way to train VAE. When training Sparse structureVAE,
TRELLIS/TRELLIS/trainers/vae/sparse_structure_vae py
z, mean, logvar = self.training_models['encoder'](ss.float(), sample_posterior=True, return_raw=True)
True is used, which is a common way to train VAE.

But when training the SparseStructureFlowModel model,

Image

Here the latent is read from the processed file, but datasets uses False when reading the encoder latent, which means only taking the mean and not re-parameterizing.

Image

So I took sample_posterior=False, got ss_latents, and then entered it into Sparse structureVAE's decoder, and got the following result

Image
When I apply sample_posterior=True, the posterior is normal

Image
I would like to ask how your SparseStructureFlowModel is trained. Is the result of sample_posterior=False used for gt? If this is gt, decoder should not be used, because the distribution fitted during training is different

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Compare TRELLIS/TRELLIS/trainers/vae/sparse_structure_vae with the dataset path used by SparseStructureFlowModel, focusing on sample_posterior=True during training versus False when reading processed latents. Determine which latent is intended as ground truth and whether decoding it should reproduce the reported behavior; done means the training and decoder expectations are clearly resolved.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.