kvcache-ai / kvcache-ai/Mooncake
[RFC]: Add Go DummyClient support and a multi-process benchmark
- Dominant language
- C++
- Stars
- 6.6k
- Forks
- 1.2k
- Avg merge
- 3d 5h
- Merged PRs (30d)
- 312
Description
## Motivation
Mooncake provides a C++ `DummyClient` that communicates with a standalone
`mooncake_client`, but the Go bindings currently lack the APIs needed to create
and configure a `DummyClient`, access the shared-memory buffers prepared during
setup, and exercise the single-object `PutFrom` path.
This RFC proposes adding the required bindings and a multi-process Go benchmark
for validating multiple independent application processes sharing one
standalone `mooncake_client`.
An implementation is available in
[kvcache-ai/Mooncake#3015](https://github.com/kvcache-ai/Mooncake/pull/3015).
It ports the relevant work from
[zchuango/Mooncake#29](https://github.com/zchuango/Mooncake/pull/29)
onto the current Mooncake `main` branch.
## Runtime Architecture
The request path is:
Go worker -> DummyClient -> RPC -> standalone `mooncake_client`
-> RealClient -> Mooncake Store
The standalone `mooncake_client` owns the `RealClient` and exposes the required
RPC handlers. Each benchmark worker is an independent operating-system process
with its own `DummyClient`, address space, and registered shared-memory context.
Separate processes are required because goroutines would still share one address
space and would not reproduce the intended multi-client shared-memory setup.
## Proposed Changes
### 1. Expose DummyClient through the Go and C bindings
Allow the Store bindings to explicitly create either a `RealClient` or a
`DummyClient`.
The proposed Go API uses:
- `New()` to create the default `RealClient`;
- `NewWithType(MOONCAKE_CLIENT_DUMMY)` to create a `DummyClient`.
The public Go API uses the Go-defined `ClientType` type and does not expose CGo
types to callers.
### 2. Preserve the existing Go Setup API
Keep the existing seven-argument Go `Setup` method for `RealClient` and add:
`DummySetup(memPoolSize, localBufferSize, serverAddress, ipcSocketPath)`
Go does not support optional function parameters. Extending the existing
`Setup` signature would therefore break existing Go call sites.
### 3. Preserve C API compatibility
Keep the existing C creation and setup APIs unchanged for current direct C
callers, and add explicit entry points for dummy clients:
- keep `mooncake_store_create()` as the default `RealClient` constructor;
- add `mooncake_store_create_with_type(client_type)`;
- keep the existing `mooncake_store_setup(...)`;
- add `mooncake_store_setup_dummy(...)`.
The Go wrapper will use the new entry points for `DummyClient`. Existing direct
C callers remain source-compatible, while the additional symbols provide the
configuration required by the Go binding.
Feedback on the exact function names and parameter layout is welcome, but this
RFC does not propose changing the signatures of the existing C APIs.
### 4. Provide access to DummyClient registered buffers
The benchmark must access the shared-memory buffers prepared during
`DummyClient` setup before calling `PutFrom` and `GetInto`.
The current implementation exposes operations to:
- obtain the number of registered buffers;
- query a buffer pointer and size by index;
- determine whether a buffer belongs to the hot cache;
- unregister the buffers.
The current implementation exposes these operations through the public Store
binding.
### 5. Complete the single-object PutFrom path
`DummyClient` already supports batch `put_from`, while the Go benchmark uses the
single-object `PutFrom` API.
The current implementation adds a dedicated `RealClient::put_from_dummy_helper` RPC. It:
1. finds the shared-memory context using the client ID;
2. translates the `DummyClient` address into a pointer valid in the standalone
`mooncake_client` process;
3. invokes the existing `RealClient` single-object `put_from` implementation.
The handler is registered in both the production `mooncake_client` and the test
RPC server.
The current proposal retains a dedicated single-object RPC. This keeps the
single-object API explicit, preserves its direct forwarding path, and reuses
the existing `RealClient` single-object implementation. Reusing the batch RPC
with a one-object request was considered, but not selected because it would
make the single-object API depend on batch semantics internally.
### 6. Add a multi-process Go benchmark
The benchmark:
- starts independent worker processes with `os/exec`;
- creates one `DummyClient` per worker;
- distributes keys among workers;
- writes deterministic data with `PutFrom`;
- reads data with `GetInto`;
- verifies the returned contents;
- reports failures, verification errors, and throughput.
Using separate processes, rather than goroutines, provides independent address
spaces, `DummyClient` instances, shared-memory contexts, and CPU affinity.
## Shared-Memory Safety
Pointers are process-local virtual addresses. `RealClient` must not directly
dereference an address received from `DummyClient`.
The RPC request includes the `DummyClient` address, size, device ID, and client
ID. `RealClient` uses the corresponding shared-memory context to translate the
dummy address into a valid local pointer.
Before accessing registered memory, the Go benchmark validates:
- conversion to the platform `int` range;
- the required size against the registered buffer capacity.
Go slices are constructed using the actual accessible buffer length.
## Scope
Included:
- C and Go `DummyClient` bindings;
- compatibility-preserving C creation and setup APIs;
- a separate Go `DummySetup`;
- access to registered `DummyClient` buffers;
- single-object `PutFrom` support;
- production and test RPC registration;
- a multi-process Go `DummyClient` benchmark.
Explicitly excluded from this proposal:
- changes to `mooncake-store/benchmarks/stress_cluster_bench.cpp`;
- CI and workflow changes;
- unrelated repository changes.
## Validation
The current implementation has been validated by:
- building `mooncake_store`, `mooncake_master`, and `mooncake_client`;
- building the existing Go `examples/basic` program;
- validating Put, Exists, GetSize, and Get through the Go binding;
- completing a DummyClient write run with 2/2 successful operations and no
failures;
- completing a DummyClient read run with 2/2 successful operations, no
failures, and no verification errors;
- rebuilding both Go examples after automated review fixes;
- confirming that the pull request contains only the intended 11 files;
- confirming successful benchmark runs return exit status 0;
- confirming operation and worker failures return a non-zero exit status;
- verifying deliberately corrupted data produces `Verify Errors: 1` and
non-zero worker and master exit statuses;
- running Rust `cargo check --locked` against the compatibility-preserving
C API;
- running `gofmt`, targeted pre-commit hooks, and the project C/C++ format
check.
These checks provide small-scale functional validation, not full performance,
high-concurrency, or stress coverage.
## Current Status
The implementation PR is ready for review. All 12 existing inline review threads
have been addressed and resolved.
## References
- Source proposal:
[zchuango/Mooncake#29](https://github.com/zchuango/Mooncake/pull/29)
- Implementation:
[kvcache-ai/Mooncake#3015](https://github.com/kvcache-ai/Mooncake/pull/3015)
Contributor guide
Research direction
Start by reviewing implementation PR #3015, then inspect the Go Store bindings, C creation/setup entry points, production and test RPC registration, and the multi-process benchmark entry point. Done means DummyClient creation and setup work without breaking existing APIs, registered buffers and single-object PutFrom operate correctly, and the documented build, benchmark, formatting, and failure-path checks pass.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, go
- Domain
- backend, distributed-systems, testing-qa
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100