Device expert slot store: ExpertSlotStore still has no virtual read, so the seam holds the concrete host type (replaces dead #1124)
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 423
- Forks
- 53
- Avg merge
- 20h 26m
- Merged PRs (30d)
- 310
Description
Row: ENG-EXPERT-STREAM-DEVICE
Replaces #1124, which the tracker no longer resolves. gh issue view 1124 returns
GraphQL: Could not resolve to an issue or pull request with the number of 1124, not a
closed state. The row's own spec still points at it as the live tracking handle
(.agents/specs/expert-stream-device-slots.md:1344: "Tracking issue: #1124, which stays
OPEN"), so the capability below currently has no owner a reader can reach.
The description is recovered verbatim from the retired index
(.agents/completed/issue-index.md:343) and then re-grounded against origin/main
23ac6f1a7, because that record is weeks old and three of its four pieces have moved.
Recovered text (verbatim, .agents/completed/issue-index.md:343)
--device cudastill cannot SERVE a larger-than-pool GGUF after #1123; it refuses by
name instead of dying mid-stream. The missing capability is a DEVICE expert slot store,
and it is four pieces:HostExpertSlotStoreis the only productionExpertSlotStore
(include/vllm/model_executor/host_expert_slot_store.h:28, the only other subclass
being a test double) whileinclude/vllm/model_executor/expert_streamer.h:8-9,30-31
claims "the production destination is a contiguous device-side slot array" and is FALSE
today; the interface has no device-capable read, becauseKqExpertSlicereads back
throughHostExpertSlotStore::Slot(), the CONCRETE class (qwen3_5.cpp:5258,5314); the
filler ispread-into-host, sinceSlotForWriteis handed straight to::pread
(expert_streamer.cpp:76-94); and the consumer is device-gated byis_cpu()at
qwen3_5.cpp:5578. Sized: 2790 slices per token at 2,490,368 B is 6.95 GB per token
against a 119.631 GiB pool already holding the dense remainder.
Grounding verdict: STILL REAL, and it is now piece 2 alone
Piece by piece against today's tree. Two of the four are fixed, and saying so is the point
of re-filing rather than copying.
Piece 1 — half fixed. DeviceExpertSlotStore now EXISTS
(include/vllm/model_executor/device_expert_slot_store.h:61, implemented in
src/vllm/model_executor/device_expert_slot_store.cpp). Nothing in production constructs
one: git grep -n 'DeviceExpertSlotStore' -- src include returns only its own declaration
and definition plus one comment. So the class landed and nothing selects it. The false
comment the record complains about is also gone — expert_streamer.h:18 now says the
opposite.
Piece 2 — STILL REAL, verbatim, and this is the whole issue now. ExpertSlotStore
declares six virtuals and none of them is a read:
virtual ~ExpertSlotStore() = default;
virtual size_t slot_bytes() const = 0;
virtual int32_t slot_count() const = 0;
virtual void WriteSlot(int32_t slot, const uint8_t* src, size_t bytes) = 0;
virtual uint8_t* SlotForWrite(int32_t slot) = 0;
virtual void CommitSlot(int32_t slot, size_t bytes) = 0;
and the seam still holds the CONCRETE host type, so it cannot be swapped:
include/vllm/model_executor/expert_stream_seam.h:153—std::unique_ptr<HostExpertSlotStore> store_;src/vllm/model_executor/expert_stream_seam.cpp:343—store_ = std::make_unique<HostExpertSlotStore>(slots, slot_bytes);src/vllm/model_executor/expert_stream_seam.cpp:163and:219—return store_->Slot(r.slot);
HostExpertSlotStore::Slot is not on the interface, which is exactly why store_ cannot
become a base pointer.
Piece 3 — FIXED. CommitSlot is on the interface (above) and ExpertStreamer calls
it, so the filler is no longer pread-into-host by construction.
Piece 4 — FIXED for unified parts, still real for discrete. The is_cpu() guard moved
out of qwen3_5.cpp and widened to
if (cpu || p.host_memory_is_device_addressable()) in
src/vllm/model_executor/expert_stream_seam.cpp:419-431. A discrete device still falls
through.
So what is owed
One thing: a virtual read on ExpertSlotStore, and ExpertStreamLane::store_ holding
the base rather than HostExpertSlotStore. The row's spec already calls this W2 and
already says W1 landed unreached because of it
(.agents/specs/expert-stream-device-slots.md:1344). This issue exists so that statement
points at a handle that resolves.
Anchors in the recovered text that have DRIFTED — do not chase them
qwen3_5.cpp:5258, :5314, :5578 and expert_streamer.cpp:76-94 are all stale.
KqExpertSlice is now a three-line forwarder to ::vllm::expert_stream::ExpertSlice.
Ground by symbol.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with ExpertSlotStore in include/vllm/model_executor/expert_slot_store.h and the store_ member and construction in expert_stream_seam.h/.cpp. Trace the Slot reads at expert_stream_seam.cpp:163 and :219, then add the virtual read seam and make ExpertStreamLane::store_ use the base type. Done means the concrete HostExpertSlotStore dependency is removed while existing host reads remain supported.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- ai-infra-agents, backend
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 70/100