Add non-interactive `aspire terminal` automation commands (capture / run / send / wait)
- Dominant language
- C#
- Stars
- 6.3k
- Forks
- 991
- Avg merge
- 2d 15h
- Merged PRs (30d)
- 196
Description
### Is there an existing issue for this?
- [x] I have searched the existing issues
### Is your feature request related to a problem? Please describe the problem.
`aspire terminal attach` hands the local console to a PTY and takes primary control. That is the wrong shape for an agent, and for a human diagnosing a stuck process:
- it is interactive and blocks,
- it registers as an HMP1 peer,
- it **drives the grid dimensions** of the session it connects to.
Follow-up to the AppHost terminal spike (#19887), which added AppHost-owned terminals (interaction-service `InputType.Terminal`, the dashboard terminal dock) alongside the existing `WithTerminal(...)` resource terminals. There is currently no non-interactive way to observe or drive either kind from the CLI.
We want a second, non-interactive family of commands that lets an agent do three things against a terminal that already exists in a running AppHost:
1. **Observe** — dump the current screen state.
2. **Act** — send text/keys.
3. **Synchronise** — wait for expected output, with a meaningful exit code.
### Describe the solution you'd like
A sketch, not a settled design — the point of filing this is to iterate on it separately.
```
aspire terminal
├── ps list terminals (exists)
├── attach interactive takeover (exists)
├── capture + dump current screen state
├── run + execute a script
├── send + sugar: one-line type/key
├── wait + sugar: one-line wait
├── record [reserved] recording
└── mouse [reserved] click / drag
```
The core is **`capture` (observe) + `run` (act & synchronise)**; `ps` discovers and `attach` remains the human path. `record`/`mouse` are namespace reservations so we don't need a breaking rename later.
**Addressing.** Today `attach` takes a resource name plus `--replica N`. That doesn't cover AppHost-owned terminals, which have no resource at all — only an id. A single `` positional would resolve `repl` (resource, replica 0) → `repl#1` (replica shorthand) → `trm_9f3a…` (terminal id). `aspire terminal ps` needs `Id` and `Kind` (`resource` | `apphost`) columns, and must enumerate `TerminalService` terminals — today `ListTerminalsAsync` only enumerates resources carrying a `TerminalAnnotation`, so AppHost-owned ids are undiscoverable.
**Exit-code contract.** A wait timeout should still emit the screen alongside the non-zero exit code (`CliExitCodes.WaitTimeout` already exists). An agent diagnosing a hang needs to see the screen that failed to match; making it choose between the exit code and the content would be hostile. `--format json` should carry content *and* metadata in one round trip so this is usable as a single agent tool call.
#### Script format: a tape-inspired dialect
The commands are predominantly for agents, so the input language should be one agents already know. Survey of what exists:
| Language | Agent familiarity | Verdict |
|---|---|---|
| **Expect (Tcl)** | Highest — *the* PTY automation language | **Trap.** Real Expect is Tcl: `set`, `if`, `proc`, `expect_before`, pattern/action lists. We can't host Tcl, so we'd accept a subset and agents would confidently emit valid Expect we reject. `spawn` is also meaningless here — the process already exists and the AppHost owns it. |
| **pexpect** | Very high | Out — a library API, not a script format. Needs Python. |
| **VHS `.tape`** | Moderate | **Leading candidate.** See below. |
| **chat(8)** | Low | Too weak — no keys, no timeouts, no capture. |
| **tmux `send-keys`** | High | Not an automation language — no waiting at all. |
| **asciinema cast** | High | A recording *format*, not automation. Relevant to `record` later. |
The applicable slice of the tape grammar:
```
Type "" Sleep
Enter Tab Space Backspace Escape Wait[+Screen|+Line][@interval] /regex/
Up Down Left Right ScrollUp ScrollDown
Ctrl[+Alt][+Shift]+ Screenshot
```
Why it's attractive:
1. **No control flow.** We'd reject a large slice — the presentation/recording surface (`Output`, `Require`, `Hide`/`Show`, `Copy`/`Paste`, `Source`, `Env`, and ~15 `Set` variants). But everything rejected is *presentation*, never *logic*. An agent cannot express control flow we're unable to run, which is exactly the Expect failure mode. Rejecting `Set Theme` is trivially explainable; rejecting `if`/`proc` is not.
2. **Already isomorphic to Hex1b.** `Type`, `Enter`, `Tab`, arrows, `Ctrl`/`Alt`/`Shift`, `ScrollUp`/`ScrollDown`, `Sleep`, `Wait` line up essentially 1:1 with `Hex1bTerminalInputSequenceBuilder` + `Hex1bTerminalInputSequence.ApplyAsync`, so parsing is close to a direct transcription. `Set TypingSpeed` and the per-command `Type@500ms` / `Wait@200ms` override syntax are meaningful for us too and map onto `SlowTypeAsync`.
3. **Covers all three verbs** — input, synchronisation, observation — in one language.
The familiarity gap versus Expect is real but cheap to close: the grammar is ~15 lines, so `run --help` can print the entire dialect and an agent one-shots it with no prior knowledge. That is not true of Expect.
### Additional context
Considerations from the design discussion that should carry forward:
**VHS has no terminal emulator — we would own emulation, and this must be stated honestly.** VHS's `go.mod` pulls `go-rod` (headless Chrome/CDP) and its README requires external `ttyd` (which serves **xterm.js**) and `ffmpeg`. It spawns a PTY, hands it to ttyd, drives a headless browser, screenshots frames, and pipes them to ffmpeg. Screen reads go through JS evaluated against xterm.js — from [`testing.go`](https://github.com/charmbracelet/vhs/blob/main/testing.go):
```go
v.Page.Eval("() => Array(term.rows).fill(0).map((e, i) => term.buffer.active.getLine(i).translateToString().trimEnd())")
```
So VHS is a *language* plus an outsourcing hack. We'd adopt only the language, and our fidelity would be strictly better — Hex1b is a real in-process emulator holding the authoritative grid. Our `Wait` would be *more* faithful than VHS's, which matches a trimmed plain-text projection with no attributes, whereas a `Hex1bTerminalSnapshot` carries full cell fidelity.
Consequence: describe this as a **tape-inspired dialect, not VHS compatibility**. Claiming compatibility we don't have is worse than declaring a dialect and printing its grammar in `--help`.
**`Screenshot` genuinely diverges.** In VHS it writes a browser PNG; ours would write text/ansi/svg/html rendered from the Hex1b grid. Same keyword, different output type — the one place borrowing the grammar could actively mislead. Options: infer format from the file extension (`.txt`/`.ansi`/`.svg`/`.html`, with `.png` possible later since Hex1b already renders SVG), or rename to `Capture` and reject `Screenshot` with a pointer.
**Non-negotiable: `capture` and `wait` must be pure observers.** They must not register as an HMP1 peer and must not resize the PTY. `attach` deliberately takes primary control and drives dimensions — a snapshot that reflowed the grid would corrupt the exact state being diagnosed, and would silently change what a concurrently-attached human sees. `send` necessarily mutates, but still must not resize. This is also why `Set Width`/`Set Height` would be rejected by the dialect.
**Implementation shape (feasibility already checked).** Two kinds of terminal behind one CLI surface:
- *AppHost-owned* (interaction dialogs, dock) live in-process in `TerminalService` and `IAspireTerminal` already exposes `GetScreenText()`, `SendTextAsync`, `SendKeyAsync`, `WaitForTextAsync`. Trivial.
- *Resource terminals* (`WithTerminal`) live in separate `Aspire.TerminalHost` processes reached over a control UDS. `TerminalHostControlProtocol` has only `getSession`/`shutdown`/`getInfo` at `ProtocolVersion = 2`; it needs screen/input/wait methods and a v3 bump. That's cheap because `TerminalReplica` already holds a live `Hex1bTerminal`.
`Hex1b.Automation` supplies the rest server-side: `Hex1bTerminalAutomator.CreateSnapshot()`, `WaitUntilTextAsync`/`WaitUntilNoTextAsync`, `AutomationStepRecord`/`StepResult` for per-step JSON reporting, and `ToAnsi()`/`ToSvg()`/`ToHtml()` so every `--format` renders host-side — **the CLI never needs a terminal emulator**, it receives finished bytes. Hex1b has no tape parser, so that piece would be ours; it's small because the language has no control flow.
A new backchannel capability (e.g. `terminals.automation.v1`) alongside `terminals.v1` would gate this so an older AppHost fails with a clean "update Aspire.Hosting" message rather than a confusing empty result.
**Open questions to settle in the follow-up:**
1. Tape-inspired dialect, or an Expect-flavoured one after all?
2. `Screenshot` — keep the keyword with extension-inferred format, or rename to `Capture`?
3. Keep `send`/`wait` sugar, or `capture` + `run` only? (Two ways to do one thing is a smell; the counter-argument is shell one-liner ergonomics.)
4. Should `ps` list AppHost-owned terminals by default — which changes existing output — or only under `--all`?
5. Should driving a terminal that already has attached peers require `--force`?
Contributor guide
Research direction
Start by reading the existing aspire terminal commands, TerminalService and IAspireTerminal, then trace TerminalHostControlProtocol and TerminalReplica for resource terminals. Review Hex1b.Automation's snapshot, wait, input, and rendering APIs before resolving the open questions about the tape-inspired dialect, Screenshot, command shape, listing, and attached peers. Done means an agreed design and implementation plan covering both terminal kinds, capability negotiation, observer semantics, formats, and exit codes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp
- Domain
- backend, cli, tooling
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 28/100