a2aproject / a2aproject/a2a-rs

a2acli: conformance — smoke-check a live agent against the A2A TCK

Ouverte
#189 0 commentaires 0 réactions 0 personnes assignées Voir sur GitHub
enhancement
Langage dominant
Rust
Étoiles
75
Forks
19
Merge moyen
11 h 27 min
PR mergées (30 j)
21

Description

Part of #181 (Tier 2). **Phase 2 — surface not yet defined upstream.**

## Summary

`A2ACLI_CONFORM_001` (Tier 2) requires a `conformance` command that smoke-checks a live agent against the A2A TCK (§7.1).

`a2acli` has no such command. The [TCK](https://github.com/a2aproject/a2a-tck) is a Python suite driven as:

```
./run_tck.py --sut-url http://localhost:9999 --category all --compliance-report report.json
```

## Why

`a2acli` is the natural place to point at an agent and ask "does this thing actually implement A2A?" — it already resolves the card, selects a transport and drives every core operation. The command is a shortcut for what a developer otherwise assembles by hand.

## Surface to agree first — and one design question that matters

The Go CLI has no `conformance` command, so there is **nothing to conform to**. The substantive question is what "against the A2A TCK" means for a Rust binary:

1. **Shell out to the TCK** — faithful to the requirement's wording and always current, but makes a Rust CLI depend on a Python toolchain being installed. For a statically-linked binary distributed via `cargo install` and Homebrew, that is a significant regression in what "works out of the box" means.
2. **Implement the smoke checks natively** — self-contained and fast, but "conformance" then means *`a2acli`'s* reading of the spec, not the TCK's, and the two will drift. Calling it conformance while not running the conformance suite is a claim the command cannot back.
3. **Detect and delegate** — run the TCK when present, otherwise run a native subset and say plainly which one ran.

(3) is the honest option, provided the output never lets the two be confused. The requirement says *smoke-check*, not *full compliance report*, which leaves room — but whichever way this goes, the command must state what it actually verified.

This is worth raising in `a2a-cli` before implementation; the answer likely belongs in the spec rather than being decided per-implementation, since a `conformance` command whose meaning varies by language defeats the purpose.

## Scope (pending the above)

- [ ] Agree upstream in `a2a-cli` what `conformance` verifies and how; record the outcome here.
- [ ] Implement `conformance` against the agreed surface.
- [ ] Output must name the check basis — TCK version, or the native subset and the spec version it was written against — so a passing report cannot be mistaken for a stronger claim than it is.
- [ ] Per-check results with the requirement each maps to; `text` and `-o json` per §11.2/§11.3.
- [ ] Exit code follows §11.6: the CLI's own success or failure, not the agent's conformance verdict, which is carried in the payload — the same decoupling as `A2ACLI_EXIT_002`. A non-conforming agent is a successful run reporting a negative result.
- [ ] Tests: a conforming fixture agent passes; a fixture with a known defect reports that specific check as failed while the command still exits 0.

## Requirements closed

| ID | Requirement |
| --- | --- |
| `A2ACLI_CONFORM_001` | `conformance` — smoke-check a live agent against the A2A TCK (§7.1) |

Guide de contribution

Ouvrir le guide de contribution

Évaluation

Cette issue n'a pas encore été évaluée.

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.