kvcache-ai / kvcache-ai/Mooncake
[Bug]: batch_get_session_start fails with 702 INVALID_REPLICA when SSD offloading is enabled
- Dominant language
- C++
- Stars
- 6.6k
- Forks
- 1.2k
- Avg merge
- 3d 5h
- Merged PRs (30d)
- 312
Description
### Bug Report
### Description
I deployed Mooncake master and client using the latest main branch, with SSD offloading enabled. When retrieving data via the Get Session API, I received error `702 (ErrorCode::INVALID_REPLICA)`. The logs show the following errors:
```
E0825 11:48:28.436453 675216 real_client.cpp:5005] No complete memory replica for key: -models-DeepSeek-V4-Flash-0731-Channel-INT8-w8a8_9ef0d3fc87769163f4a9ca7132f99420d47e3b40eb000648fcae5b788a6f0f65_tp3_cp0_pp0_deepseek_v4_c4
E0825 11:48:28.436558 675219 real_client.cpp:5005] No complete memory replica for key: -models-DeepSeek-V4-Flash-0731-Channel-INT8-w8a8_334485bcb2be8214beda12734ad46ac3ac9a8c71c7c33b81a184ce65e58df9f2_tp6_cp0_pp0_deepseek_v4_c4
E0825 11:48:28.436663 675216 real_client.cpp:5005] No complete memory replica for key: -models-DeepSeek-V4-Flash-0731-Channel-INT8-w8a8_49d5dfdaf24abeb87994a10f24550206352df5608833921098d6e74e6dcda9f3_tp3_cp0_pp0_deepseek_v4_c4
E0825 11:48:28.436770 675219 real_client.cpp:5005] No complete memory replica for key: -models-DeepSeek-V4-Flash-0731-Channel-INT8-w8a8_501666f1aa9c91c993cd32cdb569d2d6d1a01217c2c044a301cdbda2cebb40b9_tp6_cp0_pp0_deepseek_v4_c4
E0825 11:48:28.436877 675216 real_client.cpp:5005] No complete memory replica for key: -models-DeepSeek-V4-Flash-0731-Channel-INT8-w8a8_c80468b0bfe1208838299de45678e9e9c16cfc13a7715bc7394a109a60d794bf_tp3_cp0_pp0_deepseek_v4_c4
```
### Root Cause
When SSD offloading is enabled, some data resides on SSD instead of in‑memory.
The `batch_get_session_start` method only selects memory replicas. As a result, keys whose replicas exist only on SSD will return error 702.
```
std::vector RealClient::batch_get_session_start(
const std::vector &keys) {
std::vector results(
keys.size(), static_cast(toInt(ErrorCode::INVALID_PARAMS)));
if (!client_) {
LOG(ERROR) << "Client is not initialized";
return results;
}
if (keys.empty()) {
return {};
}
// Master interaction only here: query replicas + lease.
const auto query_results = client_->BatchQuery(keys);
auto local_endpoints = client_->GetLocalEndpoints();
std::lock_guard lock(session_mutex_);
for (size_t i = 0; i < keys.size(); ++i) {
if (!query_results[i]) {
results[i] = static_cast(toInt(query_results[i].error()));
get_sessions_.erase(keys[i]);
continue;
}
auto query_result = query_results[i].value();
if (query_result.IsLeaseExpired()) {
results[i] = static_cast(toInt(ErrorCode::LEASE_EXPIRED));
get_sessions_.erase(keys[i]);
continue;
}
const auto *replica =
SelectCompleteMemoryReplica(query_result.replicas, local_endpoints);
if (!replica) {
LOG(ERROR) << "No complete memory replica for key: " << keys[i];
results[i] = static_cast(toInt(ErrorCode::INVALID_REPLICA));
get_sessions_.erase(keys[i]);
continue;
}
// QueryResult members are const: erase + emplace (no operator=).
get_sessions_.erase(keys[i]);
get_sessions_.emplace(keys[i],
FilterQueryResult(query_result, *replica));
results[i] = 0;
}
return results;
}
```
### Expected behavior
Similar to `batch_get_into`, use `SelectBestReplica` to select the best replica for each key.
### Before submitting...
- [x] Ensure you searched for relevant issues and read the [documentation]
Contributor guide
Research direction
Start in real_client.cpp around the batch_get_session_start implementation and compare its replica selection with batch_get_into and SelectBestReplica. Verify the change handles keys whose replicas are on SSD and that those requests no longer return ErrorCode::INVALID_REPLICA (702).
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- backend-api-design, distributed-systems
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 78/100