a2aproject / a2aproject/a2a-tck
[Bug]: vendored specification is pinned to A2A v1.0.0, so §5.4 error mappings are two releases stale
- Lingua principale
- Python
- Stelle
- 50
- Fork
- 40
- Merge medio
- 7g 1h
- PR unite (30g)
- 1
Descrizione
### What happened?
## Summary
`specification/` is pinned to A2A **v1.0.0** and has not been refreshed since
2026-03-13. A2A released **v1.0.1** on 2026-05-28, which rewrote six of §5.4's
nine error-mapping rows. The kit's expectations in `tck/requirements/base.py`
encode the v1.0.0 values, so a server that implements the current specification
fails MUST-level checks for doing the right thing.
This is not a report of a logic bug. The tests assert the table they were given;
the table they were given has been superseded.
## Evidence
`specification/version.json` in this repository records the pin itself:
```json
{
"downloadTime": "2026-03-13T07:43:14Z",
"organization": "a2aproject",
"repository": "A2A",
"branch": "v1.0.0",
"commitHash": "173695755607e884aa9acf8ce4feed90e32727a1"
}
```
That commit is A2A's `v1.0.0` tag. `specification/specification.md` is
byte-identical to `git show v1.0.0:docs/specification.md` (md5
`65ae1635632ad20180dc78aad097ec2d`, zero differing lines), as is
`specification/a2a.proto`.
The change landed in A2A `757f0ec` ("fix(spec): recent transcoding-related error
changes", #1627) on 2026-04-14, and shipped in the `v1.0.1` tag on 2026-05-28.
`git merge-base --is-ancestor 757f0ec v1.0.0` answers no; against `v1.0.1` and
`main`, yes. The `v1.0.0` tag has not moved.
## The six rows
| Error | vendored (A2A v1.0.0) | A2A v1.0.1 and `main` |
|---|---|---|
| `TaskNotCancelableError` | `409 Conflict` | **`400 Bad Request`** |
| `PushNotificationNotSupportedError` | `UNIMPLEMENTED` | **`FAILED_PRECONDITION`** |
| `UnsupportedOperationError` | `UNIMPLEMENTED` | **`FAILED_PRECONDITION`** |
| `ContentTypeNotSupportedError` | `415 Unsupported Media Type` | **`400 Bad Request`** |
| `InvalidAgentResponseError` | `502 Bad Gateway` | **`500 Internal Server Error`** |
| `VersionNotSupportedError` | `UNIMPLEMENTED` | **`FAILED_PRECONDITION`** |
The other three rows (`TaskNotFoundError`, `ExtendedAgentCardNotConfiguredError`,
`ExtensionSupportRequiredError`) agree between the two.
## What this does to a run
Measured 2026-09-01 at `de6af18` against a server implementing the current
specification. Five of the six rows are reachable by tests today:
```
CORE-CANCEL-002 [http_json] expected 409, got 400
STREAM-SUB-003 [grpc] expected UNIMPLEMENTED, got FAILED_PRECONDITION
GRPC-ERR-002 [grpc] VersionNotSupportedError, UnsupportedOperationError,
PushNotificationNotSupportedError — all three expect
UNIMPLEMENTED, all three get FAILED_PRECONDITION
HTTP_JSON-STATUS-001 [http_json] expected 415, got 400
```
(`InvalidAgentResponseError`'s row does not appear to be asserted by any test, so
it is stale but currently invisible.)
The pattern is worth noting because it is diagnostic rather than circumstantial:
each of these requirements is graded on all three bindings and fails on **exactly
the one binding whose cell the two releases disagree about**, passing on the
others. `CORE-CANCEL-002` passes on `jsonrpc` (`-32002`) and `grpc`
(`FAILED_PRECONDITION` — a cell both releases agree on) and fails only on
`http_json`. `STREAM-SUB-003` is the mirror: it passes on `jsonrpc` and
`http_json` and fails only on `grpc`. A server returning the *wrong error* would
fail all three bindings; a stale cell in a mapping table fails one.
Note that #207 made this visible rather than causing it. Before that change, the
`CORE-CANCEL-002` and `STREAM-SUB-003` tests accepted any error, so two of these
rows were unreachable. Tightening them was correct; it just landed on stale data.
## Suggested fix
Two parts, both already routine here:
1. `make spec` — `scripts/update_spec.sh` already defaults to `--branch main`,
and the history shows this being done before ("feat: update A2A spec to
commit … and align TCK").
2. Update the six `ErrorBinding` constants in `tck/requirements/base.py`, which
are hand-maintained rather than derived from the vendored document, e.g.:
```python
TASK_NOT_CANCELABLE_ERROR = ErrorBinding(
name="TaskNotCancelableError",
jsonrpc_code=-32002,
http_status=409, # v1.0.0; v1.0.1 says 400
grpc_status="FAILED_PRECONDITION",
reason="TASK_NOT_CANCELABLE",
)
```
Since the constants and the vendored markdown can drift independently, a unit
test parsing §5.4's table out of `specification/specification.md` and
asserting it against the `ErrorBinding` set would keep step 2 from being
forgotten the next time step 1 runs.
Happy to open a PR for either or both if that is useful.
## One caveat, stated up front
A2A's own §6 says patch version numbers "do not affect protocol compatibility"
and "MUST not be considered when clients and servers negotiate protocol
versions" — yet v1.0.1 changed HTTP statuses and gRPC statuses on the wire. Both
documents therefore describe "A2A 1.0", and no peer can signal which patch it
targets. So there is a reasonable counter-argument that the underlying fault is a
wire-affecting change shipped in a patch release, and that belongs upstream in
`a2aproject/A2A` rather than here. Raising it because it seems better to name it
than to have it come up in review. Either way it does not change which text a
conformance kit should grade against: the newest released text of the version it
targets.
## Duplicate check
Searched this repository's open and closed issues for "specification", "409",
"error mapping" and "1.0.1". The nearest neighbours are #201 (vendored spec
declares GET for `/tasks/{id}:subscribe` while `SUBSCRIBE_TO_TASK_BINDING` uses
POST) and #216 (a different complaint about
`test_content_type_not_supported_error`); neither covers the snapshot being
superseded. Filing separately on that basis — happy to have it closed as a
duplicate if I missed one.
### Relevant log output
```shell
# 1. What the kit says it pinned
$ cat specification/version.json | head -8
"downloadTime": "2026-03-13T07:43:14Z",
"branch": "v1.0.0",
"commitHash": "173695755607e884aa9acf8ce4feed90e32727a1"
# 2. The vendored copy is that release, byte for byte
$ git clone -q https://github.com/a2aproject/A2A /tmp/A2A
$ git -C /tmp/A2A show v1.0.0:docs/specification.md > /tmp/v100.md
$ diff /tmp/v100.md specification/specification.md && echo IDENTICAL
IDENTICAL
# 3. The change is not in v1.0.0, and is in v1.0.1 and main
$ for r in v1.0.0 v1.0.1 main; do
printf '%-8s ' "$r"
git -C /tmp/A2A merge-base --is-ancestor 757f0ec "$r" && echo has-757f0ec || echo lacks-757f0ec
done
v1.0.0 lacks-757f0ec
v1.0.1 has-757f0ec
main has-757f0ec
# 4. The two tables, side by side
$ git -C /tmp/A2A show v1.0.0:docs/specification.md | grep -E '^\| `TaskNotCancelableError'
| `TaskNotCancelableError` | `-32002` | `FAILED_PRECONDITION` | `409 Conflict` |
$ git -C /tmp/A2A show v1.0.1:docs/specification.md | grep -E '^\| `TaskNotCancelableError'
| `TaskNotCancelableError` | `-32002` | `FAILED_PRECONDITION` | `400 Bad Request` |
# 5. Observed failures, full profile, a2a-tck@de6af18
CORE-CANCEL-002 (http_json): Expected error code 409 (TaskNotCancelableError), got 400
STREAM-SUB-003 (grpc): Expected error code UNIMPLEMENTED (UnsupportedOperationError), got FAILED_PRECONDITION
GRPC-ERR-002 (grpc): expected VersionNotSupportedError (UNIMPLEMENTED), got FAILED_PRECONDITION
HTTP_JSON-STATUS-001: expected ContentTypeNotSupportedError (415), got 400
```
### Code of Conduct
- [x] I agree to follow this project's Code of Conduct
Guida per i contributori
Apri la guida per i contributori
Direzione di ricerca
The vendored specification is in specification/specification.md and the error mappings are defined in tck/requirements/base.py. Run `make spec` to update the specification to the latest main branch. Then, update the six ErrorBinding constants (e.g., TASK_NOT_CANCELABLE_ERROR) to match the new HTTP status and gRPC status values from the updated table in §5.4. Verify by running the failing tests: CORE-CANCEL-002, STREAM-SUB-003, GRPC-ERR-002, and HTTP_JSON-STATUS-001.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Valutazione
- Stack tecnologico
- python
- Ambito
- documentation, testing
- Tipo di issue
- Bug
- Difficoltà
- 2/5
- Tempo stimato
- 1-3 ore
- Stato di attività
- Attiva
- Chiarezza
- Specificata chiaramente
- Idoneità per principianti
- 70/100