kvcache-ai / kvcache-ai/Mooncake

[Bug]: batch_get_session_start fails with 702 INVALID_REPLICA when SSD offloading is enabled

Open Beginner friendly
#3,658 3 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
C++
Stars
6.6k
Forks
1.2k
Avg merge
3d 5h
Merged PRs (30d)
312

Description

### Bug Report

### Description
I deployed Mooncake master and client using the latest main branch, with SSD offloading enabled. When retrieving data via the Get Session API, I received error `702 (ErrorCode::INVALID_REPLICA)`. The logs show the following errors:
```
E0825 11:48:28.436453 675216 real_client.cpp:5005] No complete memory replica for key: -models-DeepSeek-V4-Flash-0731-Channel-INT8-w8a8_9ef0d3fc87769163f4a9ca7132f99420d47e3b40eb000648fcae5b788a6f0f65_tp3_cp0_pp0_deepseek_v4_c4
E0825 11:48:28.436558 675219 real_client.cpp:5005] No complete memory replica for key: -models-DeepSeek-V4-Flash-0731-Channel-INT8-w8a8_334485bcb2be8214beda12734ad46ac3ac9a8c71c7c33b81a184ce65e58df9f2_tp6_cp0_pp0_deepseek_v4_c4
E0825 11:48:28.436663 675216 real_client.cpp:5005] No complete memory replica for key: -models-DeepSeek-V4-Flash-0731-Channel-INT8-w8a8_49d5dfdaf24abeb87994a10f24550206352df5608833921098d6e74e6dcda9f3_tp3_cp0_pp0_deepseek_v4_c4
E0825 11:48:28.436770 675219 real_client.cpp:5005] No complete memory replica for key: -models-DeepSeek-V4-Flash-0731-Channel-INT8-w8a8_501666f1aa9c91c993cd32cdb569d2d6d1a01217c2c044a301cdbda2cebb40b9_tp6_cp0_pp0_deepseek_v4_c4
E0825 11:48:28.436877 675216 real_client.cpp:5005] No complete memory replica for key: -models-DeepSeek-V4-Flash-0731-Channel-INT8-w8a8_c80468b0bfe1208838299de45678e9e9c16cfc13a7715bc7394a109a60d794bf_tp3_cp0_pp0_deepseek_v4_c4
```
### Root Cause
When SSD offloading is enabled, some data resides on SSD instead of in‑memory.
The `batch_get_session_start` method only selects memory replicas. As a result, keys whose replicas exist only on SSD will return error 702.

```
std::vector RealClient::batch_get_session_start(
const std::vector &keys) {
std::vector results(
keys.size(), static_cast(toInt(ErrorCode::INVALID_PARAMS)));
if (!client_) {
LOG(ERROR) << "Client is not initialized";
return results;
}
if (keys.empty()) {
return {};
}
// Master interaction only here: query replicas + lease.
const auto query_results = client_->BatchQuery(keys);
auto local_endpoints = client_->GetLocalEndpoints();
std::lock_guard lock(session_mutex_);
for (size_t i = 0; i < keys.size(); ++i) {
if (!query_results[i]) {
results[i] = static_cast(toInt(query_results[i].error()));
get_sessions_.erase(keys[i]);
continue;
}
auto query_result = query_results[i].value();
if (query_result.IsLeaseExpired()) {
results[i] = static_cast(toInt(ErrorCode::LEASE_EXPIRED));
get_sessions_.erase(keys[i]);
continue;
}
const auto *replica =
SelectCompleteMemoryReplica(query_result.replicas, local_endpoints);
if (!replica) {
LOG(ERROR) << "No complete memory replica for key: " << keys[i];
results[i] = static_cast(toInt(ErrorCode::INVALID_REPLICA));
get_sessions_.erase(keys[i]);
continue;
}
// QueryResult members are const: erase + emplace (no operator=).
get_sessions_.erase(keys[i]);
get_sessions_.emplace(keys[i],
FilterQueryResult(query_result, *replica));
results[i] = 0;
}
return results;
}

```

### Expected behavior
Similar to `batch_get_into`, use `SelectBestReplica` to select the best replica for each key.

### Before submitting...

- [x] Ensure you searched for relevant issues and read the [documentation]

Contributor guide

Open the contributing guide

Research direction

Start in real_client.cpp around the batch_get_session_start implementation and compare its replica selection with batch_get_into and SelectBestReplica. Verify the change handles keys whose replicas are on SSD and that those requests no longer return ErrorCode::INVALID_REPLICA (702).

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
backend-api-design, distributed-systems
Issue type
Bug
Difficulty
2/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
78/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.