a2aproject / a2aproject/a2a-tck
Offer: language-neutral conformance vectors for CARD-SIGN-*, currently tagged not-automatable
- Dominant language
- Python
- Stars
- 50
- Forks
- 40
- Avg merge
- 7d 1h
- Merged PRs (30d)
- 1
Description
The four `CARD-SIGN-*` requirements in `tck/requirements/agent_card.py` are tagged `NOT_AUTOMATABLE` and `backlog/tasks/task-29` records them as a coverage gap with no owner. `backlog/tasks/task-11.6` records the reasoning as "CARD-SIGN-*: non-automatable (JCS signing) - document as skip". I would like to supply what makes them automatable, and I think the reason they read as non-automatable is worth naming: testing a live SUT for signing conformance needs the SUT to hold a key and sign on demand, which is a lot to ask of a black-box harness. But the part of §8.4 that actually diverges between implementations is not the live behaviour, it is the canonical byte string, and that is a pure function with no SUT, no key and no network.
That part is testable as a static corpus, and it is worth testing because the implementations currently disagree. I signed Agent Cards with `a2a-sdk` 1.1.2 and verified them with `@a2a-js/sdk` 1.0.1 and back again: an ASCII-only card verifies in both directions, and cards carrying non-ASCII string content, a non-BMP object key, or a float in `AgentExtension.params` are rejected in both directions. Two independent RFC 8785 implementations adjudicate the Python canonicalizer as non-conformant on all three axes. Separately, neither SDK reproduces the worked example printed in §8.4.1, and they fail it identically, which is exactly the kind of agreement a live-SUT interop suite cannot detect.
What I am offering is a versioned corpus of input-to-expected-bytes vectors covering §8.4.1 canonicalization, §8.4.2 JWS construction with fixed test keys, and §8.4.3 verification, with every rejecting vector naming the clause that rejects it and the layer at which rejection occurs. The canonicalization layer needs no SUT at all and can run in this repository's CI directly. The signing and verification layers use committed test keys so that a SUT-free implementation can self-check, and can additionally drive a live SUT where one exposes a signing endpoint. It would sit beneath the existing Python requirements rather than replacing them: `CARD-SIGN-001` and `CARD-SIGN-002` become "the SUT's canonicalizer reproduces the corpus", which is a real assertion, rather than a skip.
I would ship it as a directory of JSON vectors plus a manifest, with a small runner in Python for this repository and a second runner in another language so the vectors are demonstrably not encoding one implementation's habits, and a digest over the manifest so an implementer can state which corpus revision they pass. I am happy to have the corpus live in this repository, or to host it and have this repository consume it as a pinned dependency, whichever the maintainers prefer. Before writing any of it I would like to know two things: whether vectors under `tck/` are welcome as data rather than as Python `RequirementSpec` entries, and whether you would want the §8.4.1 field-presence vectors held back until the specification resolves the rule-1 question I have raised at a2aproject/A2A#2122, since about a fifth of the canonicalization vectors depend on how that lands.
Contributor guide
Assessment
This issue has not been assessed yet.