lablup / lablup/mlxcel

feat(server): server-side MCP tool loop through the /v1/responses built-in tool contract

Open
#1,457 0 comments 0 reactions 0 assignees View on GitHub
area:architecture priority:low status:backlog type:security
Dominant language
Rust
Stars
467
Forks
54
Avg merge
4h 25m
Merged PRs (30d)
310

Description

## Context

While classifying the llama-server b10621 tool and MCP surfaces in #1435, the `--mcp-servers-config` / `--mcp-servers-json` flags and the `/tools` endpoint were marked not applicable: upstream runs the agent loop inside its browser Web UI and `tools/server/README-dev.md` declares `/tools` UI-internal, so without a UI there is no consumer. This issue records the product-level alternative that was deliberately kept out of the compatibility work. It is not a b10621 alias and must not be filed under the compatibility manifest.

## Proposal

Let `mlxcel-server` run an MCP tool loop on the server side, exposed through the OpenAI Responses API built-in tool contract that mlxcel already serves at `/v1/responses` (see `docs/responses-api.md`). A request would declare `tools: [{"type": "mcp", "server_label": ..., "server_url": ... or a server configured at startup, "allowed_tools": [...], "require_approval": ...}]`, and the server would iterate model turn, tool call, tool result until the model stops or a bound is hit, emitting the standard `mcp_list_tools`, `mcp_call`, and `mcp_approval_request` output items and their streaming events. Any OpenAI-compatible client then gets tool execution without a bespoke UI.

## Scope

- Startup-configured MCP servers only in the first iteration: a Cursor-compatible `mcpServers` JSON (same shape b10621 accepts) with stdio transport; per-request `server_url` (HTTP/SSE transport) is a follow-up behind an explicit allowlist.
- Tool discovery at startup and lazy respawn on call, with per-call timeout, output-size cap, and a maximum number of tool rounds per request.
- Approval semantics: honour `require_approval` by pausing the response with an `mcp_approval_request` item and resuming on the next request, or reject the tool when approval cannot be collected.
- Security boundary: child processes inherit server privileges, so the feature is off by default, requires an explicit flag, and refuses to start when the API is bound to a non-loopback address without API-key authentication. No generic URL proxy and no shell or filesystem built-ins are added.
- Keep `/v1/chat/completions` unchanged; tool execution is a Responses-only capability.

## Acceptance criteria

- [ ] A `/v1/responses` request with an `mcp` tool against a configured stdio server produces `mcp_list_tools`, `mcp_call`, and a final assistant message, in both streaming and non-streaming modes.
- [ ] Tool rounds, per-call time, and output size are bounded and the bounds are observable in the response usage or error.
- [ ] The feature is disabled by default and startup fails with a diagnostic when it is enabled on an unauthenticated non-loopback listener.
- [ ] A misbehaving MCP child (hang, crash, oversized output, invalid JSON-RPC) fails only its own tool call and the response completes with an error item.
- [ ] Integration tests use a scripted stdio MCP server fixture; a real-checkpoint run demonstrates a tool-using model completing a two-round task.

## References

- https://github.com/ggml-org/llama.cpp/blob/c1d0e7a004015f23bc0233470b747b596f29b264/tools/server/server-mcp.h
- https://modelcontextprotocol.io
- https://platform.openai.com/docs/guides/tools-connectors-mcp

Contributor guide

Open the contributing guide

Research direction

Start by reading docs/responses-api.md and tools/server/README-dev.md to understand the existing Responses contract and server boundaries. Then map the /v1/responses implementation and startup configuration; the issue does not name their files. Done means meeting the acceptance criteria, including bounded tool execution, safe startup checks, streaming and non-streaming coverage, and a scripted stdio MCP integration test.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
backend-api-design, security
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.