Azure / Azure/awesome-azd

template-submission workflow fails at PR creation: "GitHub Actions is not permitted to create or approve pull requests"

Open
#830 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
TypeScript
Stars
269
Forks
176
Avg merge
11h 1m
Merged PRs (30d)
37

Description

## Summary

The `template-submission.yml` workflow fails at the final **Create Pull Request** step. The submission branch is pushed successfully, but the PR is never opened.

- **Failing run**: https://github.com/Azure/awesome-azd/actions/runs/24737452045 (issue #823)
- **Error**: `##[error]GitHub Actions is not permitted to create or approve pull requests.`

Source: [`peter-evans/create-pull-request@v8`](https://github.com/Azure/awesome-azd/blob/main/.github/workflows/template-submission.yml#L170) invoked with `token: ${{ secrets.GITHUB_TOKEN }}`. The default `GITHUB_TOKEN` cannot open PRs when the org and/or repo setting *"Allow GitHub Actions to create and approve pull requests"* is disabled. Branch push succeeds; `POST /repos/{owner}/{repo}/pulls` is rejected.

## Impact

Every template submission that passes validation gets a pushed branch (e.g. `template-submission-823`) but **no PR**. Submissions silently stall and require manual intervention by a maintainer.

## Secondary bug (fix in the same PR)

Lines 129 / 135 of `template-submission.yml` write validator output to the repo root:

```yaml
if node website/scripts/validate-template.js "$SOURCE_REPO" > validation_result.json 2>validation_errors.log; then
```

`peter-evans/create-pull-request` then sweeps them into the commit:

```
3 files changed, 26 insertions(+)
create mode 100644 validation_errors.log
create mode 100644 validation_result.json
```

**Fix**: redirect to `"$RUNNER_TEMP/validation_result.json"` and `"$RUNNER_TEMP/validation_errors.log"` (and update the `cat` on line 135), or `rm -f` them before the Create Pull Request step.

---

## Options for security review

Two families of solutions:

- **A. Keep PR creation inside the Action** — needs a credential that''s allowed to open PRs.
- **B. Move PR creation out of the Action** — the Action only pushes a branch; PR creation happens somewhere else.

### A1 — Enable the repo/org setting
- **Change**: Settings → Actions → General → check *"Allow GitHub Actions to create and approve pull requests"*. Must be enabled at **both** the Azure org level and this repo.
- **Pros**: Zero code/secret changes. Keeps the ephemeral, auto-scoped `GITHUB_TOKEN`.
- **Cons**: Broadens what *every* workflow in this repo can do. Security teams often disable this org-wide on purpose.
- **Review questions**: Does Azure org policy allow this? Are we comfortable granting PR-create to every workflow, not just this one?

### A2 — Dedicated GitHub App (recommended automation path)
- **Change**: Create an App ("awesome-azd-template-bot") with permissions `contents: write` + `pull-requests: write`, installed only on `Azure/awesome-azd`. Mint a 1-hour installation token per run via [`actions/create-github-app-token`](https://github.com/actions/create-github-app-token), pass to `peter-evans/create-pull-request`.
- **Pros**: Least privilege. One repo, two permissions. 1-hour token lifetime. Distinct `awesome-azd-bot[bot]` identity in audit log. No user PAT to rotate. Org policy stays tight.
- **Cons**: Create + maintain a GitHub App; store `APP_ID` + private key as repo secrets; rotate key on a schedule.
- **Review questions**: Who owns the App? Where is the private key stored? Rotation policy?

### A3 — Personal Access Token (PAT)
- **Change**: Fine-grained PAT scoped to `Azure/awesome-azd` with `contents: write` + `pull-requests: write`, stored as secret `TEMPLATE_BOT_TOKEN`.
- **Pros**: Simplest setup.
- **Cons**: Tied to an individual''s account. Expires and must be rotated. PRs appear authored by that user. Weakest governance.
- **Review questions**: Whose account hosts it? Rotation plan? OK that PRs are authored under a human identity?

### B1 — Workflow pushes branch only; maintainer clicks "Compare & pull request" (simplest & arguably most secure)
- **Change**: Drop the `peter-evans/create-pull-request` step. After push, comment on the submission issue with the auto-generated URL: `https://github.com/Azure/awesome-azd/pull/new/template-submission-{N}`. A maintainer clicks it, reviews, opens the PR.
- **Pros**: Zero bot identity. Zero new secrets. No org policy change. Human-in-the-loop by design — the repo can''t auto-open PRs at all.
- **Cons**: Manual click per submission. Doesn''t scale at high volume.
- **Review questions**: What''s the expected submission volume? Is one click per submission acceptable?

### B2 — Label-gated follow-up workflow
- **Change**: Issue-triggered workflow pushes the branch. A second workflow triggered by a maintainer applying a `ready-for-pr` label opens the PR.
- **Pros**: Explicit human gate. Combines naturally with A2 (App only acts after label).
- **Cons**: Still needs a PR-create-capable credential somewhere (A1/A2/A3). Adds a second workflow file.

### B3 — External service opens the PR
- **Change**: Action writes submission metadata to a queue (label/artifact/external queue). A small trusted service (Azure Function, Container App, VM) picks it up and opens the PR using its own App/PAT.
- **Pros**: PR-creation credential lives outside CI entirely.
- **Cons**: Adds infrastructure. Still ends up needing an App or PAT — just hosted elsewhere. Larger attack surface overall. Rarely worth it unless a bot service already exists for this repo.

### B4 — Kill the automation entirely
- **Change**: Remove the workflow. Maintainers run a local script/`gh` to convert validated submissions into PRs.
- **Pros**: No bot identity at all. Maintainer 2FA backs every PR.
- **Cons**: Manual, slow, loses the validate-at-submission-time value.

## Comparison

| | A1 setting | A2 App | A3 PAT | B1 branch-only | B2 label-gated | B3 external service | B4 manual |
|---|---|---|---|---|---|---|---|
| Automation | full | full | full | low | medium | full | none |
| New credential | none | App key | PAT | none | depends | App/PAT + infra | none |
| Org policy change | yes | no | no | no | depends | no | no |
| Credential lifetime | n/a | 1h token | long-lived | n/a | depends | depends | n/a |
| Audit identity | generic bot | distinct bot | looks human | maintainer | maintainer + bot | distinct bot | maintainer |
| Blast radius if leaked | n/a | one repo, 1h | PAT''s scope/life | n/a | depends | depends | n/a |
| Scales with volume | yes | yes | yes | no | medium | yes | no |

## Recommendation

Two defensible endpoints depending on preference:

1. **If you want full automation with minimum new moving parts**: **A2 — dedicated GitHub App**. Strictly better than A1 or A3 on privilege scope, credential lifetime, and auditability.
2. **If you want zero new credentials at any cost**: **B1 — workflow pushes the branch, maintainer clicks the "Compare & pull request" link from an issue comment**. Arguably the most secure option in absolute terms because no bot identity that can open PRs exists at all.

**B2** (label-gated) is a nice reinforcement on top of A2 if you want explicit human gating in addition to branch protection.

## Ask

Please review A1–A3 and B1–B4 with the security team and pick a direction. Happy to send a PR implementing the chosen option plus the secondary `$RUNNER_TEMP` fix.

Contributor guide

No contributing guide indexed for this repository

Research direction

Review .github/workflows/template-submission.yml, especially the Create Pull Request step around line 170 and validator output around lines 129 and 135. First compare options A1–A3 and B1–B4 with the security team, then implement the selected PR-creation direction and move validation files to RUNNER_TEMP. Done means validated submissions no longer stall and generated validation logs are not committed.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions
Domain
ci-cd, security
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.