microsoft / microsoft/TRELLIS

Testing VAE and observed some color difference

Open
#150 0 comments 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
13.7k
Forks
1.3k
PR merge metrics
No merged PRs in 30d

Description

Hi, I tried to get the DINO features and then encode and decode latents using pre-trained VAE. I observed that there is some color difference between reconstruction and input image as shown below. Does it look fine?

Secondly, I had to adjust the code here and change latent.coords[:, 1:] -> latent.coords to make decoding work. Is this correct, do you know why it is needed

https://github.com/user-attachments/assets/e8c44f52-68ce-4ec0-ac7f-e58c290f1f64

Image

?

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in dataset_toolkits/encode_latent.py around line 118 and trace how latent coordinates are passed into encoding and decoding. Compare the reconstructed image with the input and determine whether the coordinate adjustment and color difference are expected; done means the behavior has a clear explanation or a confirmed fix.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.