aws / aws/agent-toolkit-for-aws

MCP spec conformance: 3 requirement(s) violated (via @hasmcp/mcp-spec-test) — spec 2025-11-25

Open
#265 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
2.7k
Forks
314
Avg merge
1d 1h
Merged PRs (30d)
51

Description

An automated MCP protocol conformance sweep using `@hasmcp/mcp-spec-test` found that the public AWS MCP gateway (reached via `mcp-proxy-for-aws`, at `https://aws-mcp.us-east-1.api.aws/mcp`) rejects a version-less `tools/list` request with a JSON-RPC `-32602 Invalid request parameters` error instead of serving it on the default revision, and that a stock official-SDK client cannot complete a handshake or list tools against it. This was tested against spec revision **2025-11-25**, one of two revisions this server negotiates; against the newer revision (2026-07-28) the same server fails 9 requirements, including a `server/discover` failure not seen here — see the separate issue filed for that revision, since the results differ noticeably by revision.

## Conformance report

MCP 2025-11-25 conformance report
target uvx mcp-proxy-for-aws@1.6.4 https://aws-mcp.us-east-1.api.aws/mcp
transport stdio
supported 2026-07-28, 2025-11-25

FAILED (3) — the server deviates from the spec here

Version negotiation
✗ a request with no version at all is served on the default
a version-less tools/list must be served, not refused: {"code":-32602,"message":"Invalid request parameters","data":""}

Official SDK interop
✗ a stock official-SDK client completes the handshake
MCP error -32602: Invalid request parameters
✗ a stock official-SDK client can list tools
MCP error -32602: Invalid request parameters

NOT VERIFIED (14) — skipped; a skip is not a pass

Capability methods
– tools/list returns schema-conformant tools
– tools/call on an unknown tool is an error, not a crash
– tools/call returns a schema-conformant CallToolResult
– prompts/list returns schema-conformant prompts
– prompts/get returns messages with a role and content
– resources/list returns schema-conformant resources
– resources/templates/list returns schema-conformant templates
– resources/read returns contents for every sampled resource
– resources/read on an unknown uri is an error
– following nextCursor terminates and does not repeat a page
– an invalid pagination cursor is rejected (SHOULD)
cannot determine what the target supports — initialize returned no result (status 0)

Version negotiation
– initialize returns the schema-required fields
– the handshake settles on the revision under test
no session — initialize returned no result (status 0)

Official SDK interop
– the handshake settles on a revision inside the supported window
target refused a fresh initialize: {"code":-32602,"message":"Invalid request parameters","data":""} — so which revision it negotiates could not be established

PASSED (3)

server/discover
✓ the suite is reading a schema that matches the features it selected

Version negotiation
✓ an unsupported version offered at the handshake is refused or downgraded, not echoed

Official SDK interop
✓ the official SDK does not yet implement the newest revision

Summary
3 passed 3 failed 14 not verified
20 cases applied, in 13426ms

Verdict: not conformant — 3 requirements violated.

## How to reproduce

**With Docker:**
```
docker pull hasmcp/mcp-spec-test:latest
docker run --rm hasmcp/mcp-spec-test --spec-version 2025-11-25 -c "uvx mcp-proxy-for-aws@1.6.4 https://aws-mcp.us-east-1.api.aws/mcp"
```
(or, if that image lacks a manifest for your platform: `git clone https://github.com/hasmcp/mcp-spec-test && cd mcp-spec-test && docker build -t mcp-spec-test . && docker run --rm mcp-spec-test --spec-version 2025-11-25 -c "uvx mcp-proxy-for-aws@1.6.4 https://aws-mcp.us-east-1.api.aws/mcp"`)

**Without Docker (npx):**
```
npx @hasmcp/mcp-spec-test@latest --spec-version 2025-11-25 -c "uvx mcp-proxy-for-aws@1.6.4 https://aws-mcp.us-east-1.api.aws/mcp"
```

Note: no AWS credentials are configured in the test environment; the same `-32602` on the version-less `tools/list` and handshake calls also occurs when signing is skipped entirely (`uvx mcp-proxy-for-aws@1.6.4 --skip-auth https://aws-mcp.us-east-1.api.aws/mcp`), so this does not appear to be a missing-credentials artifact.

---
This issue was generated by an automated MCP conformance sweep. Please feel free to close it if it doesn't apply or turns out to be an environment-specific artifact.

Contributor guide

Open the contributing guide

Research direction

Run the reported @hasmcp/mcp-spec-test command against the mcp-proxy-for-aws endpoint, first with Docker or npx and then with --skip-auth. Compare the handshake and version-less tools/list results with the 2025-11-25 conformance report; done means the three failed requirements pass without introducing the separately tracked failures for the newer revision.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api, backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.