GLM-5.3-Flash stages paged KV one token at a time: ~180,000 copy-plus-synchronize round trips for an 8192-token prefill

Row: MODEL-MM-GLM53-FLASH

What

GLM-5.3-Flash stages its paged KV one token at a time, and each staged row
costs a device copy plus a full queue synchronize.

src/vllm/model_executor/models/glm5_next_kv.cpp:656-682, per DSA layer, two
loops each doing one Copy per token:

for (int64_t t = 0; t < b.cached_len; ++t) {
  const int64_t off = PagedRowOffset(lblocks, b.block_size, latent_row, t);
  uint8_t* row = Staging(&span, static_cast<size_t>(latent_row) * lat_elt);
  io.Read(lat.data, static_cast<size_t>(off) * lat_elt,
          static_cast<size_t>(latent_row) * lat_elt, row);

with the indexer twin at :674-681; StoreCaches mirrors it at :766/:795.

The synchronize is by design, and correct as written (:139-150): the
staging buffer is a single reused row (Staging, :167-170, "One per
LoadCaches / StoreCaches call"), so a deferred wait would read one row while
the driver is still writing the previous one.

Scale

Measured shape, inferred arithmetic. 11 deepseek_sparse_attention layers ×
8192 tokens × 2 caches = 180,224 round trips for one prefill. At
block_size = 32 a per-block coalesced staging would be 5,632 — a ~32×
reduction. The 8192 and the 11 are read from the model; the block size is not
pinned in this file, so treat the denominator as unconfirmed.

:668 (if (!lb.has_own_indexer) continue;) exempts shared layers from the
indexer half, which can only lower the real count.

Why it is not a one-line fix

The round trip cannot be batched without restructuring the staging buffer: it is
one row wide precisely so the synchronize is cheap and correct. Coalescing means
a per-block buffer and a different correctness argument, not just moving the
Synchronize out of the loop — doing only the latter reintroduces exactly the
hazard the comment at :139-150 describes.

Class

Performance. No wrong answer; the current shape is correct. Recorded in spec
prose as "OWED, and small", which undersells it — 180k synchronizes on a prefill
is not small, and it has never been measured against the rest of the step.

Read src/vllm/model_executor/models/glm5_next_kv.cpp:139-170 and 656-682 first, then inspect the mirrored StoreCaches loops at :766 and :795. Run a GLM-5.3-Flash prefill measurement to establish the current cost; done means safely coalescing staging work without violating the single-row synchronization guarantee and validating the result against the baseline.