mudler / mudler/vllm.cpp

Tenstorrent device-PA decode consumes the KV shadow on device_current alone, so any future stale publisher corrupts decode silently

Open
#2,670 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
423
Forks
53
Avg merge
20h 26m
Merged PRs (30d)
310

Description

Row: BACKEND-TENSTORRENT-HOST-FREE-FORWARD

Symptom

Latent today. The only publisher that sets device_current = true over a
wrong device block is the short-chunk push clobber (#2669), and #2669's
repair removes that trigger. The consumption side stays unguarded:
TryPagedAttentionDeviceDecode reads sk.device_current
(src/vt/tenstorrent/tenstorrent_ops.cpp:3183) and skips EnsurePagedKvTtnn's
mirror upload whenever the flag holds, and the
TryReshapeAndCacheDeviceDecode guard does the same at
tenstorrent_ops.cpp:2790. Any future writer that publishes the flag over a
partially correct device block corrupts decode with no error path.

Why this is a separate defect from #2669

#2669 is a write-side defect: the device block is wrong when the flag is
published. This is a read-side contract: the flag is the only proof the
reader accepts. Repairing the writer does not make the reader safe. A future
publisher, such as a new staging path, a partially failed fill that still
publishes, or a capture replay that reorders pushes, re-opens the same
corruption class.

Repair direction

Tie shadow acceptance to the fill-completeness fact the writer holds: the
push records a per-block coverage stamp, and the reader checks the stamp
before it skips the upload. Keep the mirror upload as the fallback.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Read TryPagedAttentionDeviceDecode around tenstorrent_ops.cpp:3183 and TryReshapeAndCacheDeviceDecode around line 2790, then trace the push-side publication of device_current. Define how the per-block coverage stamp is recorded and checked before either reader skips EnsurePagedKvTtnn; done means incomplete shadow coverage retains the mirror-upload fallback and decode no longer accepts device_current alone.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
backend, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.