feat(server): server-side MCP tool loop through the /v1/responses built-in tool contract
- Dominant language
- Rust
- Stars
- 467
- Forks
- 54
- Avg merge
- 4h 25m
- Merged PRs (30d)
- 310
Description
## Context
While classifying the llama-server b10621 tool and MCP surfaces in #1435, the `--mcp-servers-config` / `--mcp-servers-json` flags and the `/tools` endpoint were marked not applicable: upstream runs the agent loop inside its browser Web UI and `tools/server/README-dev.md` declares `/tools` UI-internal, so without a UI there is no consumer. This issue records the product-level alternative that was deliberately kept out of the compatibility work. It is not a b10621 alias and must not be filed under the compatibility manifest.
## Proposal
Let `mlxcel-server` run an MCP tool loop on the server side, exposed through the OpenAI Responses API built-in tool contract that mlxcel already serves at `/v1/responses` (see `docs/responses-api.md`). A request would declare `tools: [{"type": "mcp", "server_label": ..., "server_url": ... or a server configured at startup, "allowed_tools": [...], "require_approval": ...}]`, and the server would iterate model turn, tool call, tool result until the model stops or a bound is hit, emitting the standard `mcp_list_tools`, `mcp_call`, and `mcp_approval_request` output items and their streaming events. Any OpenAI-compatible client then gets tool execution without a bespoke UI.
## Scope
- Startup-configured MCP servers only in the first iteration: a Cursor-compatible `mcpServers` JSON (same shape b10621 accepts) with stdio transport; per-request `server_url` (HTTP/SSE transport) is a follow-up behind an explicit allowlist.
- Tool discovery at startup and lazy respawn on call, with per-call timeout, output-size cap, and a maximum number of tool rounds per request.
- Approval semantics: honour `require_approval` by pausing the response with an `mcp_approval_request` item and resuming on the next request, or reject the tool when approval cannot be collected.
- Security boundary: child processes inherit server privileges, so the feature is off by default, requires an explicit flag, and refuses to start when the API is bound to a non-loopback address without API-key authentication. No generic URL proxy and no shell or filesystem built-ins are added.
- Keep `/v1/chat/completions` unchanged; tool execution is a Responses-only capability.
## Acceptance criteria
- [ ] A `/v1/responses` request with an `mcp` tool against a configured stdio server produces `mcp_list_tools`, `mcp_call`, and a final assistant message, in both streaming and non-streaming modes.
- [ ] Tool rounds, per-call time, and output size are bounded and the bounds are observable in the response usage or error.
- [ ] The feature is disabled by default and startup fails with a diagnostic when it is enabled on an unauthenticated non-loopback listener.
- [ ] A misbehaving MCP child (hang, crash, oversized output, invalid JSON-RPC) fails only its own tool call and the response completes with an error item.
- [ ] Integration tests use a scripted stdio MCP server fixture; a real-checkpoint run demonstrates a tool-using model completing a two-round task.
## References
- https://github.com/ggml-org/llama.cpp/blob/c1d0e7a004015f23bc0233470b747b596f29b264/tools/server/server-mcp.h
- https://modelcontextprotocol.io
- https://platform.openai.com/docs/guides/tools-connectors-mcp
Contributor guide
Research direction
Start by reading docs/responses-api.md and tools/server/README-dev.md to understand the existing Responses contract and server boundaries. Then map the /v1/responses implementation and startup configuration; the issue does not name their files. Done means meeting the acceptance criteria, including bounded tool execution, safe startup checks, streaming and non-streaming coverage, and a scripted stdio MCP integration test.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- backend-api-design, security
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100