GAP-LAB-CUHK-SZ / GAP-LAB-CUHK-SZ/ReconViaGen

Clarification on SLAT/PVC conditioner implementation vs. paper Figure 2

Open
#23 4 comments 1 reaction 0 assignees View on GitHub
Dominant language
Python
Stars
646
Forks
21
PR merge metrics
No merged PRs in 30d

Description

Hi, thanks for releasing ReconViaGen. I’m studying the training code and had a question about the local per-view condition used for SLAT Flow.

In the paper, Section 3.2 and Figure 2 seem to describe the SLAT/PVC path as using a Condition Net similar to the SS/global path: random/learnable per-view tokens are updated by cross-attention blocks over VGGT features, producing per-view token lists `T_k`. Figure 2 also shows per-view cross-attention in SLAT Flow followed by a learned/weighted fusion.

In the released code, the SLAT conditioner appears to be simpler:

- `ModulatedSLATMultiViewCond` in `trellis/models/structured_latent_flow.py` starts from DINO `image_cond`, not learned/random per-view query tokens.
- It applies four `Linear(ctx_channels -> channels) + ReLU` blocks over concatenated VGGT features and the current condition.
- I do not see self-attention or cross-attention blocks inside the SLAT conditioner, unlike the SS conditioner.
- In `trellis/modules/transformer/modulated.py`, when the SLAT DiT receives a list of per-view conditions, it seems to average per-view cross-attention outputs with `/ len(context)` rather than using a learned weighted fusion MLP as described in the paper.
- In the `v0.5` branch I noticed `fuse_blocks` are declared in `ModulatedSLATMultiViewCond`, but they do not appear to be used in `forward`.

Could you clarify whether this released implementation is the one used for the paper’s SLAT/PVC results? If so, was the simpler MLP-based SLAT conditioner chosen for stability, memory, or ease of training compared with the cross-attention Condition Net shown in the paper? Or is the paper diagram describing an earlier/internal variant?

I’m asking because the SS/global conditioner in code closely matches the paper, while the SLAT/PVC path looks architecturally different. Any explanation of the intended design tradeoff would be very helpful.

Contributor guide

No contributing guide indexed for this repository

Research direction

Read trellis/models/structured_latent_flow.py and trellis/modules/transformer/modulated.py, focusing on ModulatedSLATMultiViewCond, fuse_blocks, and per-view context fusion. Compare those paths with Section 3.2 and Figure 2 of the paper. Done means documenting whether the released SLAT/PVC implementation produced the reported results and explaining any intended architectural difference.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
computer-vision, machine-learning
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.