Tenstorrent device-PA decode consumes the KV shadow on device_current alone, so any future stale publisher corrupts decode silently
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 423
- Forks
- 53
- Avg merge
- 20h 26m
- Merged PRs (30d)
- 310
Description
Row: BACKEND-TENSTORRENT-HOST-FREE-FORWARD
Symptom
Latent today. The only publisher that sets device_current = true over a
wrong device block is the short-chunk push clobber (#2669), and #2669's
repair removes that trigger. The consumption side stays unguarded:
TryPagedAttentionDeviceDecode reads sk.device_current
(src/vt/tenstorrent/tenstorrent_ops.cpp:3183) and skips EnsurePagedKvTtnn's
mirror upload whenever the flag holds, and the
TryReshapeAndCacheDeviceDecode guard does the same at
tenstorrent_ops.cpp:2790. Any future writer that publishes the flag over a
partially correct device block corrupts decode with no error path.
Why this is a separate defect from #2669
#2669 is a write-side defect: the device block is wrong when the flag is
published. This is a read-side contract: the flag is the only proof the
reader accepts. Repairing the writer does not make the reader safe. A future
publisher, such as a new staging path, a partially failed fill that still
publishes, or a capture replay that reorders pushes, re-opens the same
corruption class.
Repair direction
Tie shadow acceptance to the fill-completeness fact the writer holds: the
push records a per-block coverage stamp, and the reader checks the stamp
before it skips the upload. Keep the mirror upload as the fallback.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Read TryPagedAttentionDeviceDecode around tenstorrent_ops.cpp:3183 and TryReshapeAndCacheDeviceDecode around line 2790, then trace the push-side publication of device_current. Define how the per-block coverage stamp is recorded and checked before either reader skips EnsurePagedKvTtnn; done means incomplete shadow coverage retains the mirror-upload fallback and decode no longer accepts device_current alone.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- backend, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100