proposal: adopt Copilot sidecar (SDK-driver) execution model

## Summary

Adopt gh-aw's **Copilot "sidecar" (SDK-driver) model** for running the Copilot
engine, replacing (or augmenting) the current one-shot
`copilot --prompt … --allow-tool … --additional-mcp-config …` CLI invocation.

Today ado-aw invokes the Copilot CLI as a single-shot command inside the AWF
sandbox:

```
/tmp/awf-tools/copilot --prompt "$(cat …/agent-prompt.md)" \
--additional-mcp-config @/tmp/awf-tools/mcp-config.json \
--allow-all-tools --allow-tool safeoutputs …
```

This binds us to the Copilot CLI's *own* MCP trust model. #1462 is a direct
symptom: to make the compiler-generated SafeOutputs MCP usable under
`--allow-all-tools`, we had to (a) mark generated MCPG entries
`isDefaultServer: true` via a `jq` step and (b) add an explicit
`--allow-tool safeoutputs`. Every future quirk of the CLI's flag/trust surface
lands on us the same way.

## How gh-aw does it (current `main`)

gh-aw does **not** run Copilot as a one-shot command in its SDK path. Instead:

1. **Copilot CLI runs as a long-lived headless sidecar**, started/stopped by a
harness:
```
copilot --headless --no-auto-update --port [--add-dir … --disable-builtin-mcps …]
```
(`actions/setup/js/copilot_sdk_sidecar.cjs` → `buildCopilotSDKServerArgs`,
`startCopilotSDKServer`.)

2. **A Node "driver" drives one session over the `@github/copilot-sdk` npm
package.** `copilot_sdk_driver.cjs` → `copilot_sdk_session.cjs` connects to
the sidecar's `COPILOT_SDK_URI` with a per-run `COPILOT_CONNECTION_TOKEN`,
reads the prompt file, runs a single programmatic session, and streams JSONL
events. Prompt/tools/model are set through SDK API calls, not CLI flags.

3. **Tool permissions are enforced by a JS `onPermissionRequest` callback**, not
the CLI trust model. The engine serializes the allow list to
`GH_AW_COPILOT_SDK_SERVER_ARGS` (JSON); `copilot_sdk_permissions.cjs` builds
the handler. `--allow-all-tools` → `{ allowAllTools: true }` → `approveAll`,
which approves *every* request including MCP server calls. No
`isDefaultServer` and no explicit safeoutputs grant are required — the
concept simply does not appear anywhere in gh-aw source.

4. **The harness adds retry/reliability logic** around the session (transient
CAPIError 400, auth-failure token refresh, `--continue` resume in CLI mode,
MCP-policy-blocked detection, permission-denial counting, etc.).

Relevant gh-aw files:
- `actions/setup/js/copilot_sdk_sidecar.cjs` — spawns headless CLI server
- `actions/setup/js/copilot_sdk_driver.cjs` — SDK-driven session entry point
- `actions/setup/js/copilot_sdk_session.cjs` — session runner + JSONL events
- `actions/setup/js/copilot_sdk_permissions.cjs` — `onPermissionRequest` handler
- `actions/setup/js/copilot_harness.cjs` — retry / reliability wrapper
- `pkg/workflow/copilot_engine_tools.go` — allow-list computation (CLI mode)
- gh-aw pins Copilot CLI `1.0.68` and `@github/copilot-sdk` `1.0.5`
(`pkg/constants/version_constants.go`)

## Why this is worth doing for ado-aw

- **Removes CLI-trust coupling.** Permission decisions move into our own JS
callback, so quirks like #1462 (`isDefaultServer`, explicit safeoutputs grant
under `--allow-all-tools`) disappear — we approve MCP calls programmatically.
- **Programmatic control of the session.** Structured JSONL events instead of
scraping CLI stdout; cleaner hooks for audit/telemetry, permission-denied
accounting, and missing-tool reporting.
- **Reliability.** A harness gives us a natural home for retries (transient
400s, auth refresh, MCP-policy-blocked detection) that are awkward around a
one-shot invocation.
- **Convergence with gh-aw.** We already mirror gh-aw's security model; aligning
the Copilot execution mechanism reduces future drift and lets us borrow their
hardening.

## Constraints / open questions specific to ado-aw

The naive port is non-trivial because our runtime shape differs from gh-aw:

1. **AWF sandbox boundary.** Today the *agent's copilot command* is the only
thing wrapped by AWF (`awf … -- ''` in
`agentic_pipeline.rs::run_agent_step`). A sidecar model means we run a
**headless Copilot server + a Node driver**. We need to decide what lives
inside the AWF L7 allowlist boundary vs. outside it, and how the driver
reaches the sidecar (localhost port inside the sandbox?). This must not widen
the network egress surface.
2. **Node/SDK availability in the sandbox.** We'd need `@github/copilot-sdk`
(pinned) plus a Node driver bundle available at runtime. Natural fit for the
existing `scripts/ado-script/` ncc-bundle mechanism (a new `copilot-sdk-driver`
bundle shipped in `ado-script.zip`), mirroring gh-aw's `actions/setup/js`.
3. **SafeOutputs MCP wiring.** Our SafeOutputs MCP already runs as a host
process reached via `host.docker.internal:8100`. Under the SDK model the
driver's permission callback replaces `--allow-tool safeoutputs` /
`isDefaultServer`, but the MCP *config* handed to the headless server still
needs the MCPG/host URLs. Confirm the headless server consumes the same
`mcp-config.json` (or an SDK-registered equivalent).
4. **BYOK / multi-provider.** gh-aw's driver treats multi-provider BYOK as the
only supported mode (`GH_AW_COPILOT_SDK_MULTI_PROVIDER_JSON`). We need to map
our `engine:` model/provider config onto the SDK's provider shape.
5. **Scope of change.** Decide whether to (a) fully replace the one-shot CLI
path, or (b) add SDK mode behind a flag and keep CLI mode as fallback (gh-aw
keeps both). Option (b) is lower risk.

## Proposed plan (incremental)

1. **Spike:** stand up a `copilot --headless --port …` sidecar + a minimal Node
driver bundle in one target (standalone) behind an opt-in
`engine.copilot-sdk: true` flag; keep the existing one-shot path as default.
2. **Permission callback:** port `onPermissionRequest` semantics so
`--allow-all-tools` and per-tool allow-lists are enforced in JS; retire the
`isDefaultServer` `jq` mutation and the forced `--allow-tool safeoutputs`
from #1462 in the SDK path.
3. **Reliability harness:** add retry/auth-refresh/MCP-policy-blocked handling.
4. **Audit/telemetry:** consume the JSONL event stream in place of stdout
scraping.
5. **Roll out across targets** (1es/job/stage) and flip the default once stable.

## Acceptance criteria

- A SafeOutputs workflow runs end-to-end via the sidecar/SDK model with **no**
`isDefaultServer` mutation and **no** forced `--allow-tool safeoutputs`.
- AWF network isolation is preserved (no new egress; documented boundary for the
sidecar + driver).
- Existing one-shot CLI path remains available (at least during rollout).
- Version pins for Copilot CLI + `@github/copilot-sdk` are centralized and
documented.

## References

- ado-aw #1462 — `fix(compile): trust generated SafeOutputs MCP for Copilot`
(the workaround this proposal would obviate)
- gh-aw `actions/setup/js/copilot_sdk_*.cjs`, `copilot_harness.cjs`
- gh-aw `pkg/constants/version_constants.go` (Copilot CLI `1.0.68`,
`@github/copilot-sdk` `1.0.5`)

Start at agentic_pipeline.rs::run_agent_step to understand the current AWF-wrapped Copilot invocation, then inspect the scripts/ado-script bundle mechanism and the referenced gh-aw sidecar, driver, session, permissions, and harness files. A completed spike should run an opt-in sidecar/SDK workflow while preserving AWF isolation, the existing CLI fallback, centralized version pins, and the SafeOutputs acceptance criteria.

Written by the indexing model from the issue text.