kvcache-ai / kvcache-ai/Mooncake

[Bug]: DummyClient::register_buffer fails for buffers outside pre-mapped shared memory segments

Open
#2,501 2 comments 0 reactions 1 assignee Claimed by @HyeonjeCho View on GitHub
bug stale
Dominant language
C++
Stars
6.6k
Forks
1.2k
Avg merge
3d 5h
Merged PRs (30d)
312

Description

### Bug Report

## Summary

When using SGLang with EAGLE speculative decoding and Mooncake SSD offload (L3 HiCache), the `DummyClient::register_buffer()` returns `-1` for EAGLE's DRAFT KV pool buffer because it is allocated outside the shared memory segments that mooncake_client pre-mapped during initialization. This makes EAGLE + Mooncake L3 combination completely unusable.

## Environment

- **SGLang**: v0.5.13
- **Mooncake Transfer Engine (CUDA13)**: v0.3.11.post1
- **CUDA**: 13.0 (Driver 535.129.03)
- **GPU**: 8x H100 (TP=8)
- **Model**: Qwen3.5-397B-A17B-FP8
- **mooncake_client mode**: standalone (DummyClient), `--threads 8`
- **SSD**: Local PCIe NVMe (`/kv-cache-volume`)

## Steps to Reproduce

1. Start mooncake_master and mooncake_client (standalone mode)
2. Start SGLang server with EAGLE speculative decoding + HiCache + Mooncake storage backend:
```
--speculative-algorithm EAGLE
--speculative-num-steps 3
--speculative-eagle-topk 1
--speculative-num-draft-tokens 4
--enable-hierarchical-cache
--hicache-storage-backend mooncake
--hicache-storage-backend-extra-config '{"standalone_storage": true, "client_server_address": "127.0.0.1:50052"}'
```
3. SGLang crashes during initialization when registering the DRAFT pool buffer

## Expected Behavior

`register_buffer` should accept buffers that are not within pre-mapped shared memory segments, either by dynamically registering a new segment or by allowing the buffer to be registered without segment constraints.

## Actual Behavior

SGLang crashes during initialization when registering the DRAFT pool buffer:

```
[2026-06-16 11:13:33 TP4] Failed to register buffer, error code: -1
[2026-06-16 11:13:33 TP4] Scheduler hit an exception: Traceback (most recent call last):
File "/sgl-workspace/sglang/python/sglang/srt/managers/scheduler.py", line xxx, in xxx
...
File "/sgl-workspace/sglang/python/sglang/srt/mem_cache/storage/mooncake_store/mooncake_store.py", line 637, in register_mem_host_pool_v2
super().register_buffer(host_pool.kv_buffer)
File "/sgl-workspace/sglang/python/sglang/srt/mem_cache/storage/mooncake_store/mooncake_store.py", line 294, in register_buffer
ret_code = self.store.register_buffer(ptr, size)
File "/sgl-workspace/sglang/python/sglang/srt/mem_cache/storage/mooncake_store/mooncake_store.py", line 297, in register_buffer
raise RuntimeError(
RuntimeError: Failed to register buffer to Mooncake Store, error code: -1
[2026-06-16 11:13:33] Received sigquit from a child process. It usually means the child failed.
[2026-06-16 11:13:33 TP5] Failed to register buffer, error code: -1
[2026-06-16 11:13:33 TP5] Scheduler hit an exception: Traceback (most recent call last):
...
RuntimeError: Failed to register buffer to Mooncake Store, error code: -1
[2026-06-16 11:13:33] Received sigquit from a child process. It usually means the child failed.
[2026-06-16 11:13:33 TP7] Failed to register buffer, error code: -1
[2026-06-16 11:13:33 TP7] Scheduler hit an exception: Traceback (most recent call last):
...
RuntimeError: Failed to register buffer to Mooncake Store, error code: -1
[2026-06-16 11:13:33] Received sigquit from a child process. It usually means the child failed.
```

Multiple TP ranks fail simultaneously (TP4, TP5, TP7 in this case). The mooncake_client log shows no errors — RPC operations (PutStart, GetReplicaList, Ping) all succeed. The failure is isolated to the `register_buffer` call.

### Before submitting...

- [x] Ensure you searched for relevant issues and read the [documentation]

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.