MP mode (LMCacheMPConnector) ignores save_decode_cache and stores decode KV, causing unbounded L1/L2 writes and premature eviction

**Label**
LMCache Onboarding (#1882) · MP mode · bug

**Summary**
In MP mode, the vLLM-side LMCacheMPConnector stores every full chunk of prompt + generated tokens with no save_decode_cache gate and no cap at the prompt length. The chunk that straddles the prompt→output boundary is completed during decode by sampled (non-deterministic) output tokens, so it hashes differently on every request. The result is a brand-new KV chunk stored per request even when the prompt set is fixed and fully cached — so the cache keeps writing after warmup, L1 fills and evicts, and shared L2 (e.g. S3) fills up. The documented save_decode_cache=false (which is the default) is honored by the in-process LMCacheConnectorV1 but not by LMCacheMPConnector.

**Details**
- The store decision runs in vLLM's vendored connector vllm/distributed/kv_transfer/kv_connector/v1/lmcache_mp_connector.py (factory maps "LMCacheMPConnector" here; no kv_connector_module_path is needed to trigger it). The editable lmcache_mp_connector.py is not the code that runs.
- Offending logic in LMCacheMPRequestMetadata.GetStoreMetadata:
_computed_blocks = tracker.num_scheduled_tokens // vllm_block_size + max(
tracker.num_vllm_hit_blocks, tracker.num_lmcache_hit_blocks)
min_available_blocks = min(len(tracker.block_hashes),
len(tracker.allocated_block_ids), computed_blocks)
num_staging_blocks = min_available_blocks - tracker.num_stored_blocks
num_chunks = num_staging_blocks // blocks_in_chunk # stores EVERY full chunk
token_ids = list(tracker.all_token_ids) # includes generated tokens_
- Trigger condition: with r = templated_prompt_len mod chunk_size, a decode chunk is written whenever r + OSL >= chunk_size; the number stored per request is floor((r + OSL) / chunk_size). For Llama-3.1 chat (template overhead ≈ 34 tokens) with chunk_size=256, OSL=250 writes 1 unique chunk/request; OSL=100 writes 0.
- lmcache.skip_save per-request (kv_transfer_params) does not help: it is not honored by the MP connector (the MP config docs explicitly warn against relying on it), and even where honored it disables all saving (prefill too).
- Versions checked: reproduced on vllm==0.26.0; source of the latest release v0.29.0 is identical (same GetStoreMetadata, no gate) — so it is not fixed upstream.

**Steps / Reproduction (if applicable)**
- Start an MP server: lmcache server --host 0.0.0.0 --port 6555 --chunk-size 256 --l1-size-gb 60 --eviction-policy LRU --eviction-trigger-watermark 1.0.
- Start vLLM with --kv-transfer-config '{"kv_connector":"LMCacheMPConnector","kv_role":"kv_both","kv_connector_extra_config":{"lmcache.mp.server_urls":["tcp://localhost:6555"]}}' (Llama-3.1-8B-Instruct).
- Replay a fixed set of prompts (e.g. aiperf ... --random-seed 42 --num-dataset-entries 50 --endpoint-type chat --isl 8192 --isl-stddev 0 --osl 250 --extra-inputs '{"ignore_eos": true, "min_tokens": 250}') so the working set fits in L1 and no eviction should occur.
- Observe: after the first pass the server keeps logging Stored 256 tokens at ~request rate, L1 memory usage 1.00 above watermark … triggering eviction fires, and external prefix-cache hit rate plateaus below 100%. With --osl 100 (so r+OSL < 256) the writes stop after the first pass and no eviction occurs — confirming the boundary/decode-chunk mechanism.

**Expected Outcome / Goal**
With save_decode_cache=false (the default), MP mode should not store decode KV. A fully-cached, replayed prompt set should be reads-only after the first pass regardless of OSL — no per-request stores, no eviction. save_decode_cache should be honored in MP mode (or the MP connector should cap stores at the prompt length by default)

**Actual Outcome (if applicable)**
MP mode stores the decode/boundary chunk on every request (unbounded, unique writes), fills L1 and shared L2, and triggers eviction even when the unique working set easily fits — because the MP connector stores all full chunks of prompt + generated tokens and ignores save_decode_cache.

Start in vllm/distributed/kv_transfer/kv_connector/v1/lmcache_mp_connector.py and inspect LMCacheMPRequestMetadata.GetStoreMetadata, especially its store decision and token range. Reproduce with the listed MP server and vLLM configuration, then verify that save_decode_cache=false prevents decode or boundary-chunk writes and that repeated prompts remain reads-only without eviction.

Written by the indexing model from the issue text.