aws-samples / aws-samples/sample-autonomous-cloud-coding-agents

fix(agent): preflight GitHub token write capability in setup_repo — a scope-less PAT fails only at push, after all work is done

Open
#801 1 comment 0 reactions 0 assignees View on GitHub
agent-runtime approved bug P1 security
Dominant language
TypeScript
Stars
143
Forks
46
Avg merge
3d 9h
Merged PRs (30d)
20

Description

## Problem

A GitHub token that authenticates successfully but **lacks write capability** passes every stage of task setup and fails only at the first `git push` — after the agent has done all of its work. The work is committed inside the microVM, never reaches the remote, and is lost when the container exits.

Observed on a real run against `aws-samples/sample-abca-playground` (task `01M0ZH0GXG3TEGE272TKM6KGG9`): a classic PAT (`ghp_`) with **zero scopes**. The run burned **69 turns / $1.90 / 7 minutes**, of which ~60 turns (≈85%) were the agent trying to work around the failed push. The files it wrote (`barcelona-guide.html`, an edit to `europe-region.html`) do not exist on the remote — no branch, no PR, no content.

## Why this credential shape is hard to diagnose

The failure mode is unusually opaque, which is what makes a preflight worth it:

- `gh api repos// --jq .permissions` returns **`push: true`** — but that field describes the *authenticated user's collaborator role*, not the *token's* capability. The account can push; the credential cannot.
- `X-OAuth-Scopes` on any API response is **empty**, which is the only real signal, and nothing surfaces it.
- `gh api rate_limit` returns 5000/hr, so the token is genuinely authenticated.
- Against a **public** repo, all reads succeed (a scope-less classic PAT retains anonymous read), so `gh repo clone` works fine.
- Every write returns **404, not 403** (GitHub avoids leaking resource existence), so no error message ever says "you lack a scope".

The agent in the observed run did eventually diagnose it correctly from `X-OAuth-Scopes`, but only after ~20 turns of probing, and it had no way to recover.

## Current behavior

Nothing validates write capability at any point:

- `agent/src/repo.py:11` `setup_repo()` clones and configures the `gh` credential helper as the git credential source. It never verifies the token can write.
- `agent/src/config.py:30` `resolve_github_token()` validates that a token *exists* (Secrets Manager or env), never that it is fit for purpose.
- A repo-wide grep for `X-OAuth-Scopes` / scope validation across `agent/src/` and `cdk/src/handlers/` returns nothing.

So a non-writing token passes provisioning → passes clone → passes the entire coding phase → fails at push.

## Proposed fix

Add a **write-capability preflight** to `setup_repo()` (`agent/src/repo.py`), immediately after the clone and remote/credential-helper configuration, and fail the task fast with an actionable message.

Either mechanism works; the second is the stronger signal:

1. Read `X-OAuth-Scopes` from an authenticated API response and assert the token carries write capability.
2. `git push --dry-run origin HEAD:refs/heads/` — exercises the real credential path end to end without mutating the remote.

Option 2 is preferred because it tests the actual code path that later fails, and it is agnostic to token type (classic vs. fine-grained expose capability differently — a fine-grained PAT has no `X-OAuth-Scopes` at all, so a scopes-only check must not treat "no scopes header" as failure for that type).

The error must name the remedy, not just the symptom — e.g. which token type the repo needs and which permission is missing, pointing at the existing decision table in `agent/README.md` ("GitHub PAT — Minimal Permissions").

## Acceptance criteria

- [ ] `setup_repo()` verifies the resolved GitHub token can write to `config.repo_url` before the agent phase begins.
- [ ] A token that cannot write fails the task **immediately** (seconds, not turns) with a `FAILED` terminal status and an actionable error naming the missing permission and the token type required.
- [ ] The check correctly passes a **fine-grained** PAT with `Contents: read+write` (i.e. absence of `X-OAuth-Scopes` is not treated as failure).
- [ ] The check correctly passes a **classic** PAT with `repo`.
- [ ] The check correctly **fails** a classic PAT with no scopes — the observed case.
- [ ] A read-only workflow (`ensure_pr` strategy `resolve`, e.g. `coding/pr-review-v1`) is **not** gated by the preflight: it never pushes, so requiring write capability there would be a regression.
- [ ] The probe does not mutate the remote (no stray branches/refs left behind) and does not write the token to disk — the existing comment at `agent/src/repo.py:48` explains why credentials must stay out of `.git/config`; the preflight must preserve that property.
- [ ] Unit tests cover pass (both token types), fail (no scopes), and read-only-bypass paths.

## Notes / scope

- Keep this **general**: a capability preflight any ABCA deployment benefits from. No playground-specific or repo-specific constants.
- The token provisioning itself (rotating the playground secret to a `repo`-scoped classic PAT) is a separate operational fix and not in scope here — this issue is about the platform detecting the condition instead of discovering it 18 turns in.
- Related but distinct: the same run **reported `success` with no PR and no pushed branch**. That is a separate finalize-gate defect, filed separately; a preflight makes the condition rare but does not make the finalize path correct.

Contributor guide

Open the contributing guide

Research direction

Start with setup_repo() in agent/src/repo.py and read resolve_github_token() in agent/src/config.py, including the credential-safety comment around line 48. Check the token permission decision table in agent/README.md and the ensure_pr resolve strategy before adding unit coverage for both writable token types, an unscoped classic token, and the read-only bypass. Done means writable workflows fail fast with an actionable error without mutating the remote or persisting credentials.

Written by the indexing model from the issue text.

Assessment

Tech stack
git, github, python
Domain
authentication, devops, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
58/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.