LTX-2.5 VAE: upstream's bf16 noise draw is a DIFFERENT sequence, and mirroring it invalidates every captured render digest
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 423
- Forks
- 53
- Avg merge
- 20h 26m
- Merged PRs (30d)
- 310
Description
Row: -
Owed by .agents/specs/ltx25-a24-video-vae-bf16.md (A24 wave 3), which is
drafted but not yet landed; this issue is filed ahead of it so the measurement is
not lost with the draft. Found while specifying wave 3 of gap A24.
This is a product decision, not a mirror question. It is filed for the
developer, and no row implements it until it is answered.
The measurement
Upstream's video VAE draws its noise from a different sequence at bfloat16
than at float32. Measured by executing Lightricks/LTX-2 @ fd4ded7f
(conv_video_decoder.py:288-294 into resnet.py:115):
- 1151 of 1152 drawn values differ between the two dtypes.
- Only 9 of 64 bf16 draws appear anywhere in the first 256 f32 draws.
So it is not a rounding of the f32 stream. It is a different stream.
Why it is a decision
Gap A24 mirrors upstream's single model dtype: distilled.py:109 sets
self.dtype = torch.bfloat16 and hands it to every component. Waves 1 and 2
(the text tower, 8e582a5f9, and the connector, #2753) mirrored it exactly,
because AGENTS.md says to mirror vLLM wherever it defines behaviour and never to
ask how a mirrored feature must behave.
Wave 3 cannot follow that rule silently. Mirroring the bf16 noise draw would
change every captured render digest in this repository — the goldens, the
committed frame digests, and the adherence-campaign renders that
.agents/specs/ltx25-adherence-rescore.md established are bit-deterministic and
whose control arm is byte-identical across 454 commits.
That is a scale of record invalidation a row should not choose on its own.
What a decision needs to answer
- (a) Mirror the bf16 draw, and re-capture every affected digest in the same
change, recording the invalidation explicitly. - (b) Keep the f32 noise stream as a named, recorded divergence from
upstream, with the reason and the measurement above, so the digests stay
comparable across the campaign. - (c) Mirror it but gate the VAE arm on a fresh set of digests captured
after the switch, leaving pre-existing renders anchored to the old stream and
marked as such.
Unverified
Whether the shipped bench geometries produce more than one temporal group, which
decides whether the tiled-blend path (a separate 31%-of-values effect measured in
the same sweep) is reachable in practice. It needs a real-weights render and a
GPU lease; none was taken.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Read .agents/specs/ltx25-a24-video-vae-bf16.md, distilled.py:109, and the upstream conv_video_decoder.py:288-294 and resnet.py:115 measurements. The work is blocked on choosing and recording one of the three noise-stream policies; if mirroring is selected, affected render digests and the adherence-rescore records must be addressed, while the tiled-blend path remains unverified without a real-weights GPU render.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, pytorch
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100