aws-samples / aws-samples/sample-autonomous-cloud-coding-agents
fix(agent): preflight GitHub token write capability in setup_repo — a scope-less PAT fails only at push, after all work is done
- Dominant language
- TypeScript
- Stars
- 143
- Forks
- 46
- Avg merge
- 3d 9h
- Merged PRs (30d)
- 20
Description
## Problem
A GitHub token that authenticates successfully but **lacks write capability** passes every stage of task setup and fails only at the first `git push` — after the agent has done all of its work. The work is committed inside the microVM, never reaches the remote, and is lost when the container exits.
Observed on a real run against `aws-samples/sample-abca-playground` (task `01M0ZH0GXG3TEGE272TKM6KGG9`): a classic PAT (`ghp_`) with **zero scopes**. The run burned **69 turns / $1.90 / 7 minutes**, of which ~60 turns (≈85%) were the agent trying to work around the failed push. The files it wrote (`barcelona-guide.html`, an edit to `europe-region.html`) do not exist on the remote — no branch, no PR, no content.
## Why this credential shape is hard to diagnose
The failure mode is unusually opaque, which is what makes a preflight worth it:
- `gh api repos// --jq .permissions` returns **`push: true`** — but that field describes the *authenticated user's collaborator role*, not the *token's* capability. The account can push; the credential cannot.
- `X-OAuth-Scopes` on any API response is **empty**, which is the only real signal, and nothing surfaces it.
- `gh api rate_limit` returns 5000/hr, so the token is genuinely authenticated.
- Against a **public** repo, all reads succeed (a scope-less classic PAT retains anonymous read), so `gh repo clone` works fine.
- Every write returns **404, not 403** (GitHub avoids leaking resource existence), so no error message ever says "you lack a scope".
The agent in the observed run did eventually diagnose it correctly from `X-OAuth-Scopes`, but only after ~20 turns of probing, and it had no way to recover.
## Current behavior
Nothing validates write capability at any point:
- `agent/src/repo.py:11` `setup_repo()` clones and configures the `gh` credential helper as the git credential source. It never verifies the token can write.
- `agent/src/config.py:30` `resolve_github_token()` validates that a token *exists* (Secrets Manager or env), never that it is fit for purpose.
- A repo-wide grep for `X-OAuth-Scopes` / scope validation across `agent/src/` and `cdk/src/handlers/` returns nothing.
So a non-writing token passes provisioning → passes clone → passes the entire coding phase → fails at push.
## Proposed fix
Add a **write-capability preflight** to `setup_repo()` (`agent/src/repo.py`), immediately after the clone and remote/credential-helper configuration, and fail the task fast with an actionable message.
Either mechanism works; the second is the stronger signal:
1. Read `X-OAuth-Scopes` from an authenticated API response and assert the token carries write capability.
2. `git push --dry-run origin HEAD:refs/heads/` — exercises the real credential path end to end without mutating the remote.
Option 2 is preferred because it tests the actual code path that later fails, and it is agnostic to token type (classic vs. fine-grained expose capability differently — a fine-grained PAT has no `X-OAuth-Scopes` at all, so a scopes-only check must not treat "no scopes header" as failure for that type).
The error must name the remedy, not just the symptom — e.g. which token type the repo needs and which permission is missing, pointing at the existing decision table in `agent/README.md` ("GitHub PAT — Minimal Permissions").
## Acceptance criteria
- [ ] `setup_repo()` verifies the resolved GitHub token can write to `config.repo_url` before the agent phase begins.
- [ ] A token that cannot write fails the task **immediately** (seconds, not turns) with a `FAILED` terminal status and an actionable error naming the missing permission and the token type required.
- [ ] The check correctly passes a **fine-grained** PAT with `Contents: read+write` (i.e. absence of `X-OAuth-Scopes` is not treated as failure).
- [ ] The check correctly passes a **classic** PAT with `repo`.
- [ ] The check correctly **fails** a classic PAT with no scopes — the observed case.
- [ ] A read-only workflow (`ensure_pr` strategy `resolve`, e.g. `coding/pr-review-v1`) is **not** gated by the preflight: it never pushes, so requiring write capability there would be a regression.
- [ ] The probe does not mutate the remote (no stray branches/refs left behind) and does not write the token to disk — the existing comment at `agent/src/repo.py:48` explains why credentials must stay out of `.git/config`; the preflight must preserve that property.
- [ ] Unit tests cover pass (both token types), fail (no scopes), and read-only-bypass paths.
## Notes / scope
- Keep this **general**: a capability preflight any ABCA deployment benefits from. No playground-specific or repo-specific constants.
- The token provisioning itself (rotating the playground secret to a `repo`-scoped classic PAT) is a separate operational fix and not in scope here — this issue is about the platform detecting the condition instead of discovering it 18 turns in.
- Related but distinct: the same run **reported `success` with no PR and no pushed branch**. That is a separate finalize-gate defect, filed separately; a preflight makes the condition rare but does not make the finalize path correct.
Contributor guide
Research direction
Start with setup_repo() in agent/src/repo.py and read resolve_github_token() in agent/src/config.py, including the credential-safety comment around line 48. Check the token permission decision table in agent/README.md and the ensure_pr resolve strategy before adding unit coverage for both writable token types, an unscoped classic token, and the read-only bypass. Done means writable workflows fail fast with an actionable error without mutating the remote or persisting credentials.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- git, github, python
- Domain
- authentication, devops, testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 58/100