google / google/capsem

perf: revisit MCP tool-call concurrency ceiling

Open
#94 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
72
Forks
13
Avg merge
1d 2h
Merged PRs (30d)
5

Description

## Summary

Non-blocking performance follow-up for MCP/tool-call concurrency. This is not a 1.3 release blocker: the current path is functional and faster than the old ~4k rps plateau, but it still does not scale cleanly.

## Current benchmark

Run from `release/1.3-cleanup-pr-v2` with `capsem-bench mcp-load`.

| concurrency | rps | p50 | p95 | p99 | errors |
|---:|---:|---:|---:|---:|---:|
| 1 | 1193.0 | 0.8ms | 1.1ms | 1.5ms | 0 |
| 10 | 5900.9 | 1.6ms | 2.4ms | 2.9ms | 0 |
| 50 | 7109.5 | 6.6ms | 9.8ms | 12.9ms | 0 |
| 200 | 7470.3 | 25.4ms | 36.0ms | 42.4ms | 0 |

## Path under test

Guest fastmcp client -> `/run/capsem-mcp-server` -> framed vsock -> capsem-process MCP endpoint -> capsem-mcp-aggregator -> builtin MCP tool -> response path.

## Why revisit

The path is pipelined and no longer obviously broken, but c=50/c=200 mostly plateau. We should isolate whether the ceiling is stdio relay, framed-vsock writer serialization, aggregator stdio, builtin tool server serialization, or telemetry/logging await cost.

## Guardrails

- Preserve the unified tool-call ledger: users care about tools, not MCP internals.
- Keep aggregator DB-free.
- Preserve security/CEL enforcement semantics.
- Keep the benchmark hermetic and include p50/p95/p99 plus CPU/thread counters.

Contributor guide

Open the contributing guide

Research direction

Start by running `capsem-bench mcp-load` from `release/1.3-cleanup-pr-v2` and compare p50, p95, p99, CPU, and thread counters across concurrency levels. Trace the listed path from `/run/capsem-mcp-server` through framed vsock, the capsem-process MCP endpoint, capsem-mcp-aggregator, and the builtin MCP tool to isolate the ceiling. Done means identifying the limiting stage while preserving the unified ledger, DB-free aggregator, security/CEL semantics, and hermetic benchmark.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
backend-api-design, performance
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.