Carry per-group slot mappings into multi-KV forwards
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 423
- Forks
- 53
- Avg merge
- 20h 26m
- Merged PRs (30d)
- 310
Description
Row: MODEL-MM-deepseek-v4-deepseek-v4-for-causal-lm
DeepSeek V4 Vision W4 found that StepInputs::slot_mapping[g] is computed for every KV group, but the runner forwards only the full-attention group's mapping through CommonAttentionMetadata. MultiKvCacheIndex carries per-group block tables but no parallel per-group slot mappings.
That makes a production multi-cache model validate the real cache groups and then either invent identity slots or ignore non-primary cache writes. W4 needs the actual SWA, compressed-latent, attention-compressor, indexer-key, and indexer-compressor slots.
Fix in the same flow: add a borrowed per-group slot-mapping channel to the shared multi-KV index, populate it from runner-owned step inputs for the forward lifetime, and mutation-prove a non-identity mapping through the registered DeepSeek V4 path. The pull request for #2411 closes this issue too.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Trace StepInputs::slot_mapping through CommonAttentionMetadata and MultiKvCacheIndex, starting with the shared multi-KV forward flow. Determine how runner-owned mappings live for the forward and how the registered DeepSeek V4 path consumes them. Done means each KV group receives its actual mapping, and a non-identity mapping is mutation-proven through that path.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- ai-infra-agents, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100