mudler / mudler/vllm.cpp

A8: the spatiotemporal latent upsampler is a product decision, not a port — its output shape has no consumer above 8 frames

Open
#2,584 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
423
Forks
53
Avg merge
20h 26m
Merged PRs (30d)
310

Description

Row: -

Owed by .agents/specs/ltx25-upsampler-arms.md under ## Owed. Split out of
#2577, which ports the dims=2
arm and is closed by that work; this one must outlive it.

A8 of .agents/specs/ltx25-completion-scope.md §4: the spatiotemporal latent
upsampler
, spatial_upsample AND temporal_upsample. Refused at
src/vllm/model_executor/models/ltx2_upsampler.cpp through
Ltx2RefuseUnportedPipelineFeature(kSpatiotemporalUpsampler).

This is a decision, not a port

The operator is small. model/upsampler/model.py:55-59 builds
Conv3d(mid, 8*mid) + PixelShuffleND(3), and :109-113 routes it through the
same first-frame drop the temporal arm already uses. The golden generator already
constructs the module, so an upstream gate is available for the asking.

The consumers are what keeps it out. It returns [c, 2f-1, 2h, 2w], and both
spatial call sites require the frame count back unchanged:

Call site Requirement
src/vllm/multimodal/ltx2_video.cpp:3521 (video latent) up.frames == vshape.frames, checked at :3525-3531
src/vllm/multimodal/ltx2_video.cpp:3548 (generated keyframe slots, dfr_pipeline.py:348) up_slots.frames == slot_positions.size(), checked at :3552-3563

They agree only when vshape.frames == 1. That is
(frames - 1) / factors.time + 1 == 1 at :3411 with factors.time 8, so
frames <= 8 — reachable, but the one configuration where the doubled frame
axis is cancelled by the mandatory drop before anything observes it.

So porting the operator would trade a named refusal for a generic shape
complaint on every clip longer than eight frames
— the substitution the guard
at ltx2_video.cpp:3473-3502 was written to prevent, and its comment says so.

What a decision needs to answer

Upstream computes this arm, and AGENTS.md says mirror it where it computes.
Against that, this tree deliberately built a diagnosis upstream does not have.
Either:

  • port the operator and accept the shape complaint, letting the existing check
    speak; or
  • port it and add a phase guard naming the arm, which leaves the operator
    ported-and-undriven and so needs a named owner and an ## Owed entry under
    Nothing lands dead; or
  • leave it refused and record the reason as ratified rather than provisional.

.agents/specs/ltx25-upsampler-arms.md §5 carries the derivation.

Note on the scope document

.agents/specs/ltx25-completion-scope.md:371 still calls A8 "the single
cheapest real gap and the only reachable unported-feature refusal", and §8 sizes
it S beside A9. That is right about the operator and did not count the second
consumer at :3548. That document is operator-owned and is not edited from
here; the contradiction is recorded in the row spec instead.


Line anchors re-derived at branch head: this row's own load-time guard added 16 lines above every ltx2_video.cpp site cited here, so the original numbers were stale by exactly 16.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with .agents/specs/ltx25-upsampler-arms.md §5 and inspect the refusal in src/vllm/model_executor/models/ltx2_upsampler.cpp, then trace the shape checks at src/vllm/multimodal/ltx2_video.cpp:3521 and :3548. Done means choosing one of the three stated policies and recording the decision, with any required phase guard, owner, or ## Owed entry documented.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, python
Domain
ai, backend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.