a2aproject / a2aproject/a2a-rs

a2acli: conformance — smoke-check a live agent against the A2A TCK

Offen
#189 0 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
enhancement
Vorherrschende Sprache
Rust
Sterne
75
Forks
19
Ø Merge
11 Std. 27 Min.
Gemergte PRs (30 T.)
21

Beschreibung

Part of #181 (Tier 2). **Phase 2 — surface not yet defined upstream.**

## Summary

`A2ACLI_CONFORM_001` (Tier 2) requires a `conformance` command that smoke-checks a live agent against the A2A TCK (§7.1).

`a2acli` has no such command. The [TCK](https://github.com/a2aproject/a2a-tck) is a Python suite driven as:

```
./run_tck.py --sut-url http://localhost:9999 --category all --compliance-report report.json
```

## Why

`a2acli` is the natural place to point at an agent and ask "does this thing actually implement A2A?" — it already resolves the card, selects a transport and drives every core operation. The command is a shortcut for what a developer otherwise assembles by hand.

## Surface to agree first — and one design question that matters

The Go CLI has no `conformance` command, so there is **nothing to conform to**. The substantive question is what "against the A2A TCK" means for a Rust binary:

1. **Shell out to the TCK** — faithful to the requirement's wording and always current, but makes a Rust CLI depend on a Python toolchain being installed. For a statically-linked binary distributed via `cargo install` and Homebrew, that is a significant regression in what "works out of the box" means.
2. **Implement the smoke checks natively** — self-contained and fast, but "conformance" then means *`a2acli`'s* reading of the spec, not the TCK's, and the two will drift. Calling it conformance while not running the conformance suite is a claim the command cannot back.
3. **Detect and delegate** — run the TCK when present, otherwise run a native subset and say plainly which one ran.

(3) is the honest option, provided the output never lets the two be confused. The requirement says *smoke-check*, not *full compliance report*, which leaves room — but whichever way this goes, the command must state what it actually verified.

This is worth raising in `a2a-cli` before implementation; the answer likely belongs in the spec rather than being decided per-implementation, since a `conformance` command whose meaning varies by language defeats the purpose.

## Scope (pending the above)

- [ ] Agree upstream in `a2a-cli` what `conformance` verifies and how; record the outcome here.
- [ ] Implement `conformance` against the agreed surface.
- [ ] Output must name the check basis — TCK version, or the native subset and the spec version it was written against — so a passing report cannot be mistaken for a stronger claim than it is.
- [ ] Per-check results with the requirement each maps to; `text` and `-o json` per §11.2/§11.3.
- [ ] Exit code follows §11.6: the CLI's own success or failure, not the agent's conformance verdict, which is carried in the payload — the same decoupling as `A2ACLI_EXIT_002`. A non-conforming agent is a successful run reporting a negative result.
- [ ] Tests: a conforming fixture agent passes; a fixture with a known defect reports that specific check as failed while the command still exits 0.

## Requirements closed

| ID | Requirement |
| --- | --- |
| `A2ACLI_CONFORM_001` | `conformance` — smoke-check a live agent against the A2A TCK (§7.1) |

Beitragsleitfaden

Beitragsleitfaden öffnen

Rechercherichtung

The issue is part of a larger effort (#181) and requires upstream agreement on the command's design before implementation. The a2acli command structure is in the Rust project; examine existing commands in src/cli/. The TCK is a Python suite at github.com/a2aproject/a2a-tck. Research how to shell out to Python or implement native checks. The output must follow JSON/text formatting per spec sections 11.2/11.3 and exit codes per 11.6. Start by reading the linked TCK and existing CLI code to understand the integration surface.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Bewertung

Tech-Stack
python, rust
Bereich
cli, testing
Issue-Typ
Feature
Schwierigkeit
4/5
Geschätzter Aufwand
3-5 Tage
Aktivitätsstatus
Aktiv
Klarheit
Größtenteils klar
Anfängerfreundlichkeit
45/100

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.