mudler / mudler/vllm.cpp

LTX-2.5 adherence gap is not localized: nothing separates a divergent latent from a divergent VAE decode

Open
#2,514 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
423
Forks
53
Avg merge
20h 26m
Merged PRs (30d)
310

Description

Row: LTX25-ADHERENCE-BISECT

LTX25-PROMPT-ADHERENCE W3 measured our LTX-2.5 render against the pinned CLIP
instrument and it FAILS S1: mean 35.2719 against a bound of 36.0087 recomputed
from the reference's own 25 frames, margin -0.7368. The reference reads 38.1278.
.agents/specs/ltx25-prompt-adherence.md ## Owed states that no row owns
closing that gap
, and that naming an owner "needs the attribution first".

This issue owns ONE step of that attribution, and it is the step that halves the
search space before anyone reads a pixel statistic:

Is the divergence in the LATENT, or in the VAE DECODE?

  • If our final video latent matches the oracle's and only the decoded pixels
    differ, the defect is in the video VAE decode path and the denoise loop is
    exonerated.
  • If the latent already diverges, the defect is in the denoise loop
    scheduler, sigma schedule, guidance/STG application, RoPE, or the DiT forward —
    and the VAE is exonerated.

Nothing in the tree currently distinguishes these. Every LTX-2.5 absolute
comparison this repository has taken is on PIXELS
(tests/parity/goldens/ltx2_oracle/ holds an mp4 and 25 PPM frames and no
latent), and pixels are the composition of both halves. That is precisely why
the gap is unlocalized.

The obvious experiment does not work, and saying so is part of this issue.
Running both engines at seed 42 and differencing the two final latents is
uninformative BY CONSTRUCTION. .agents/specs/ltx25-oracle-absolute.md under
## Risks/decisions already records why: "Same prompt, same seed integer,
different engine, so the sampler's noise is not the same draw." Two different
noise draws through two CORRECT denoisers produce two different latents, so a
large elementwise difference is what a healthy pair would also show, and the
measurement could not tell the two cases apart. A latent difference is only
attributable once the noise is matched, which is a separate and larger change.

The noise-independent experiment is a CROSS-DECODE, and it is what this
issue runs. Dump the ORACLE's own final video latent, then decode that one
latent through BOTH VAEs:

  • the oracle's VAE decode of it is the committed reference render, already in
    the tree and digest-verified;
  • our VAE decode of the SAME latent is the new measurement.

Both arms are then scored on the pinned CLIP instrument. The answer lands in
the units of the gap itself:

  • our VAE decode of the oracle's latent scores near the reference's 38.1278 →
    our VAE is faithful, and the divergence is in the LATENT, i.e. the
    denoise loop: scheduler, sigma schedule, guidance/STG, RoPE, or the DiT
    forward;
  • it scores near our own 35.2719 → the VAE decode accounts for the gap;
  • it lands between → the split is quantified rather than argued.

This is a PAIRED test on byte-identical input, so it needs no noise matching
and no second render of ours.

Scope, and what this issue does NOT own:

  • It owns the LATENT-versus-PIXEL split, measured by running both engines at the
    reference's own pinned request and comparing the final latents directly.
  • It does NOT own the repair. Localizing is this issue; fixing is the next one.
  • It does NOT own the frequency-domain analysis of the two existing frame sets.
    That is #2513 (LTX25-ADHERENCE-DETAIL-LOSS), which owns the pixel-statistics
    question and needs no GPU.

The request is the manifest's, byte for byte, and identical to both neighbours':

prompt   "A red fox walks slowly through a snowy pine forest at sunrise, cinematic."
320x192, 25 frames, 8 inference steps, seed 42
the four BF16 checkpoints of tests/parity/goldens/ltx2_oracle/ltx2_oracle_manifest.json

Latent tokens = (320/32)*(192/32)*ceil(25/8) = 240.

This issue also owes the error bar that the S1 verdict does not have. Our
side of every number above is ONE render. scripts/ltx25-render-confirm.sh:551
is [ "$i" = 1 ] || rm -f "$D"/frame_*.ppm and the whole compare block is
guarded by if [ "$i" = 1 ] at :471, so renders 2 and 3 are deleted by design.
The -0.7368 margin therefore has no known run-to-run spread, and a reading that
moved by 0.74 between runs would make the verdict a coin toss rather than a
finding. Every render taken under this issue retains and scores its frames, and
records loadavg and GPU clock beside each. If the S1 gap turns out to sit
inside run-to-run noise, that result outranks any bisection and is reported
first.

Related: #1854 (the question), #2295 (LTX25-PROMPT-ADHERENCE, which measured
the gap), #2513 (the pixel-side diagnosis).

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with tests/parity/goldens/ltx2_oracle/ltx2_oracle_manifest.json, .agents/specs/ltx25-oracle-absolute.md, and scripts/ltx25-render-confirm.sh around lines 471 and 551. Run the pinned request while retaining repeated frames, loadavg, and GPU-clock data, then perform the oracle-latent cross-decode and score both arms with the pinned CLIP instrument. Done means the run-to-run error bar is reported and the gap is localized to latent divergence, VAE decode, or a quantified split.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
machine-learning, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.