stacklok / stacklok/mecatl

OpenAI Responses calls materialize invalid optional MCP arguments

Open
#1,711 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Go
Stars
152
Forks
16
Avg merge
14h 48m
Merged PRs (30d)
536

Description

Bug description

When using an OpenAI-compatible Responses endpoint, a model-generated call to a directly exposed MCP tool included every optional property with minimum, empty, or first-enum values—even though those properties were irrelevant or explicitly mutually exclusive. After the tool rejected the call and the user explicitly identified the invalid property, the next run emitted the same malformed calls again.

This appears specific to Mecatl's request/replay shape: the same endpoint has been used through other agent harnesses for several days without similar tool-argument behavior.

Steps to reproduce

The observed session ID is 50028801afa1b8a77186921f8e171d66.

  1. Run mecated with the OpenAI Responses provider against an OpenAI-compatible endpoint and expose the GitHub MCP server directly.
  2. Use model gpt-5.6-terra.
  3. Ask the model to add an issue comment with mcp__github__add_issue_comment.
  4. Observe the generated arguments.

The MCP schema requires owner, repo, and issue_number. It defines these optional properties:

  • body: comment content
  • comment_id: comment ID for a reaction; its description says to omit it when targeting the issue and that it cannot be combined with body
  • reaction: required only when body is absent

The model emitted calls shaped like:

{
  "body": "...",
  "comment_id": 1,
  "issue_number": 877,
  "owner": "stacklok",
  "reaction": "+1",
  "repo": "dockyard"
}

Five calls failed with comment_id cannot be combined with body. The user then sent:

all those failed because you're adding comment_id: 1 (and probably other unnecessary fields) for no earthly reason

The immediately following run emitted five new calls with new call IDs but the same invalid comment_id: 1 and reaction: "+1" arguments. Similar calls elsewhere in the session materialized unrelated optional fields as fields: [] and sha: "".

Expected behavior

Optional MCP arguments should be omitted unless the requested operation needs them. After receiving a validation error and an explicit user correction, the model should issue corrected calls containing only body, issue_number, owner, and repo.

Actual behavior

Optional properties were populated with minimum, empty, or first-enum values. The exact malformed shape was repeated after both tool validation errors and an explicit correction, and the run stopped at the consecutive-tool-failure limit. No comments were created by either failed batch.

Environment

  • OS/version: macOS
  • mecatl version: v0.0.38-57-g7cf4e5dfd
  • Deployment shape: local mecated
  • Provider protocol: OpenAI Responses through an OpenAI-compatible endpoint
  • Model: gpt-5.6-terra
  • GitHub MCP server exposed through ToolHive group discovery

Additional context

Current code evidence narrows the issue to upstream generation influenced by Mecatl's request shape, or a much less likely SDK decoding defect:

  • provider/openai/request.go passes each tool's JSON Schema through and deliberately leaves strict unset.
  • provider/openai/request.go performs stateless full-history replay, including prior function calls, tool results, and encrypted reasoning items.
  • The persisted conversation orders the five failed tool results before the user's correction.
  • provider/openai/stream.go consumes response.output_item.done and copies item.Arguments.OfString directly into json.RawMessage.
  • In openai-go v3.61.0, the function-call arguments value is decoded as an opaque string; the SDK does not parse or default its inner JSON properties.
  • internal/adapter/mcp/tool.go forwards the decoded argument map to the MCP server unchanged.
  • engine/agent/dispatch.go records the original call arguments before any hook rewrite.

The endpoint transports Responses SSE frames without transforming response bodies, but no historical raw body capture exists for this session. A controlled reproduction should capture the raw request and SSE response with secrets removed, then compare:

  1. Mecatl's exact request versus a working harness request to the same endpoint and model.
  2. Non-strict schemas versus normalized strict-compatible schemas.
  3. A fresh single-turn request versus Mecatl's stateless replay after failed parallel calls.
  4. Replay with and without encrypted reasoning continuity.

Potential fixes may include provider-aware strict-schema normalization, safer handling of schemas with mutually exclusive optional fields, or a diagnostic mode that can capture redacted request/response payloads for controlled reproductions.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with provider/openai/request.go and provider/openai/stream.go, then trace argument forwarding through internal/adapter/mcp/tool.go and engine/agent/dispatch.go. Capture redacted request and SSE payloads and compare fresh, replayed, strict-schema, and reasoning-continuity cases. Done means identifying whether Mecatl or the endpoint creates the optional fields and applying a validated fix.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
api, backend
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.