Unify internal IPC and VSOCK codecs with bounded framing and serde_bytes
- Dominant language
- Rust
- Stars
- 72
- Forks
- 13
- Avg merge
- 1d 2h
- Merged PRs (30d)
- 5
Description
## Problem
A protocol inventory done for #207 found five codecs on Capsem's inter-process paths. Two internal ones have sharp edges:
- **Service/CLI ↔ VM owner IPC uses `tokio-unix-ipc` 0.4 with bincode 1.**
- The frame header is native-endian with an unbounded length, so a peer can force an allocation of up to 4 GiB.
- Header and payload go out as two syscalls, so every writer has to funnel through a single task.
- Enum variant order is wire ABI. `SCHEMA_HASH` hashes source bytes, so even a comment edit changes it.
- bincode cannot deserialize `serde_json::Value`. `McpToolStatus.annotations: Option` (`crates/capsem-proto/src/ipc.rs`) is a latent failure; #207 fixes that one field.
- **Guest VSOCK and the aggregator use rmp-serde, but `Vec` has no `serde_bytes`.** Bytes are encoded as a msgpack int array of up to 2× size, so "fits in a frame" can only be decided by encoding first (`crates/capsem-proto/src/lib.rs`).
- **Docs name the wrong codecs.** Service IPC is described as msgpack, the aggregator as NDJSON, and the guest MCP path as NDJSON.
## Proposal
- Move host IPC to a bounded, explicitly-endian length-prefixed rmp framing, reusing the Hello codec that already exists, and make the maximum frame size explicit.
- Add `serde_bytes` to every byte payload on rmp paths.
- Keep variant-stability tests and version negotiation.
- Benchmark IPC and VSOCK framing before and after (`crates/capsem-proto/benches/vsock_framing_micro.rs`, plus a new IPC bench).
## Out of scope
The public stream API (`capsem.stream.v1`) and REST JSON, both defined in #207.
Contributor guide
Research direction
Start by reading the existing Hello codec and the protocol definitions in crates/capsem-proto/src/ipc.rs and crates/capsem-proto/src/lib.rs. Run crates/capsem-proto/benches/vsock_framing_micro.rs and inspect the variant-stability and version-negotiation tests before designing the new IPC benchmark. Done means bounded explicit-endian rmp framing, serde_bytes on rmp byte payloads, preserved stability and negotiation tests, updated codec documentation, and before/after framing benchmarks.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- distributed-systems, networking
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100