The LTX-2.5 text tower computes a [T, 262144] lm_head, a SoftCap and a full download that nothing reads, twice per render
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 423
- Forks
- 53
- Avg merge
- 20h 26m
- Merged PRs (30d)
- 310
Description
Row: - (listed under ## Owed in .agents/specs/ltx25-device-arm-survey.md)
Found in flow by LTX25-DEVICE-ARM-SURVEY while establishing the text tower's
device-arm shape.
Gemma4Model::ForwardHiddenStates returns hidden_states and logits, and
ForwardBody (src/vllm/model_executor/models/gemma4.cpp:733-756) computes the
tied lm_head over EVERY row unless logits_indices is non-empty and shorter
than T (:738-739).
src/vllm/model_executor/models/ltx2_text_encoder.cpp:1166 calls it WITHOUT
logits_indices, so do_gather is false. The call site then reads only
run.hidden_states (:1167-1187). The [T, 262144] f32 buffer, the
vt::MatmulBT against the [262144, 3840] tied embedding table, the SoftCap
and the full Download into out.logits are all discarded.
This runs twice per render — the positive prompt at ltx2_video.cpp:2501 and
the negative one at the guiders pass — on the host, on the default
ti2vid_one_stage path.
Sizing (arithmetic, NOT a measurement)
At the shipped tower geometry (tests/vllm/models/ltx2_gemma4_text_config.json:
hidden_size = 3840, vocab_size = 262144, num_hidden_layers = 48,
intermediate_size = 15360) the lm_head is 2 * 3840 * 262144 = 2.01 GFLOP per
token against roughly 2.16e10 FLOP per token for the 48 decoder layers —
about 8.5% of the tower's GEMM work.
Against the 82.336 s of measured tower on the reference render (rc job
93a60151, n = 3, conditioning.tower 27.014 + guiders.tower 55.322, wall
302.954 s) that is of order 6 to 7 s, ~2.2% of the wall, plus a
T * 262144 * 4-byte allocation and download per pass.
No measurement was taken. This is arithmetic against a measured leaf and must
not be quoted as a measured saving.
The change, and why it is not made yet
Passing logits_indices = {0} at the one call site removes it, and it cannot
change the conditioning: capture(dnorm) (gemma4.cpp:731) records the final
hidden state BEFORE the lm_head.
The gate is the open question. A test can hold the seam
(out.logits.size() == vocab rather than T * vocab), but holding the LTX call
site itself needs an instrument this tree does not expose, and a mutation that
moves nothing is not evidence of anything. That gate design is why this is filed
rather than squeezed into the survey row.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with src/vllm/model_executor/models/ltx2_text_encoder.cpp:1166-1187 and Gemma4Model::ForwardBody in src/vllm/model_executor/models/gemma4.cpp:733-756. Inspect the existing model tests and tests/vllm/models/ltx2_gemma4_text_config.json, then determine how to instrument the LTX call site without changing conditioning. Done means a test verifies the reduced logits shape and demonstrates that the unused lm_head work is avoided.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- machine-learning, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100