port(KV-OFFLOAD): the offload identity is never parallelism-agnostic, so a cache cannot be reread under a different TP degree
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 423
- Forks
- 53
- Avg merge
- 20h 26m
- Merged PRs (30d)
- 310
Description
Row: KV-OFFLOAD
Found by re-deriving upstream a8f296083f (vllm#49858) against this tree in
#2632, wave PORTQ-1.
The gap
FileMapper and the offload cache identity both exist here
(include/vllm/v1/kv_offload/fs_io.h:86,
include/vllm/v1/kv_offload/cache_identity.h), but the mechanism the commit
widens does not. CacheIdentity carries tp_size / pp_size / pcp_size /
dcp_size / rank unconditionally in its digest
(include/vllm/v1/kv_offload/cache_identity.h:139-143), and the on-disk path
always embeds the rank —
include/vllm/v1/kv_offload/fs_io.h:97, <base>_r<rank>/<hhh>/<hh>_g<group>/<hash>.bin.
parallel_agnostic appears only in an explanatory comment at
cache_identity.h:46-47.
So a cache written under one TP degree can never be read under another, whatever
the layout.
Already recorded
.agents/specs/kv-persistence-lmcache.md:133 is feature row 50 and already
carries this: "parallel_agnostic folder collapsing … file_mapper.py:46-48,85-96
… MISSING". This issue gives that row an owner and a tracked number; it does
not re-discover it.
Upstream
file_mapper.py @ a8f296083f (a forward reference past the pin 5559679229):
:38,64-67 add a replicated_layout constructor argument written into the hashed
fields dict only when true, so existing paths keep their identity, and
:100-104 widen parallel_agnostic to
parallel.is_parallelism_agnostic or config.replicated_layout.
Size
The commit's own delta is ~10 lines: one optional identity field plus one
predicate widening. It is unreachable until the prerequisite lands.
parallel_agnostic gating, with the MLA exclusion and its four cases, is roughly
60-120 lines plus tests; upstream's tests/v1/kv_offload/test_file_mapper.py is
the executable specification, already cited at
.agents/specs/kv-persistence-lmcache.md:407.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
After the prerequisite parallel-agnostic mechanism lands, start with include/vllm/v1/kv_offload/cache_identity.h and fs_io.h, then compare the upstream file_mapper.py change at a8f296083f. Use tests/v1/kv_offload/test_file_mapper.py as the executable specification, including the MLA exclusion and four cases. Done means replicated layouts preserve existing identities while enabling the intended parallel-agnostic cache paths.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- backend, performance
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100