Azure / Azure/azure-quickstart-templates

commit-generated-on-merge skips commit-back step because head-SHA lookup misses the issue_comment workflow run

Open
#14,765 0 comments 0 reactions 1 assignee Claimed by @ouldsid View on GitHub
Dominant language
Bicep
Stars
14.9k
Forks
16.2k
Avg merge
6d 21h
Merged PRs (30d)
6

Description

## Summary

After @ouldsid's #14754 fix landed in ca5c5299, I ran #14764 as the upstream-branch canary to verify the full auto-gen path end-to-end (the verification couldn't complete on #14762 because that PR was on a fork branch). Everything worked through `/validate`:

- `adx-deployment-validation` succeeded on PR head `29608c273722b763fd7ed94a8d04214767353948`.
- The `generated-azuredeploy-14764-29608c27...` artifact uploaded by `selected-pipeline` had the correct repo-relative path (`quickstarts/microsoft.keyvault/key-vault-certificate-create/azuredeploy.json`) — confirming the #14754 staging-directory fix works as designed.

**But on merge, `commit-generated-on-merge` ran to "success" without actually committing anything.** Steps 4–7 (locate artifact, checkout, download, commit) all skipped. Master is left without `azuredeploy.json` in the sample folder.

## Root cause

In the `Resolve artifact source run` step (lines roughly 2440–2460 of `ValidateSampleDeployments.yml`), the lookup is:

```bash
RUN_ID=$(gh api \
"/repos/${REPO}/actions/runs?head_sha=${HEAD_SHA}&event=issue_comment&status=success&per_page=20" \
--jq "[.workflow_runs[] | select(.name == \"${WORKFLOW}\")] | sort_by(.run_started_at) | last | .id // \"\"")
```

with `HEAD_SHA=${{ github.event.pull_request.head.sha }}` — i.e. `29608c273722b763fd7ed94a8d04214767353948`.

The problem: for `issue_comment`-triggered workflow runs, GitHub Actions registers the run against **the default branch HEAD at the time the comment was posted**, not against the PR's head SHA. Verified via `gh run list`:

```
2026-05-13T19:44:12Z 25822348426 issue_comment sha=f6946a30 success ← THE successful /validate run
2026-05-13T19:43:39Z 25822320414 issue_comment sha=f6946a30 skipped
2026-05-13T19:41:20Z 25822203265 issue_comment sha=f6946a30 skipped
```

All `issue_comment` runs targeting #14764 are registered against `f6946a30` (master HEAD at the time), not the PR head `29608c27`. So the lookup by `head_sha=29608c27` returns zero results, the step logs `"No successful issue_comment run of 'Validate PR (Summarize + ADX)' found for 29608c27..."`, and the subsequent commit-back steps all skip.

The artifact itself is still findable by its name (`generated-azuredeploy-${PR_NUM}-${PR_HEAD_SHA}`), but the workflow uses run-SHA lookup as a first hop, not artifact-name search.

## Suggested fix

Look up the artifact by **name** rather than by run-SHA. The artifact name encodes both the PR number and the PR head SHA. Either:

1. List all artifacts in the repo filtered by name pattern:
```bash
ARTIFACT_ID=$(gh api "/repos/${REPO}/actions/artifacts?per_page=100" \
--jq "[.artifacts[] | select(.name == \"generated-azuredeploy-${PR_NUM}-${HEAD_SHA}\") | select(.expired == false)] | sort_by(.created_at) | last | .id // \"\"")
```
Then proceed straight to download with the artifact ID.

2. Or, in the run-lookup, drop the `head_sha=` filter and instead pass `?check_suite_id=...` or filter by `pull_requests[].number == ${PR_NUM}` client-side. The `workflow_runs` API entries do carry a `pull_requests` array with the PR number for issue_comment-triggered runs.

Option 1 is more direct because it avoids re-deriving the run identity at all — the artifact name is the contract.

## Severity

**Silent regression**: same shape as #14754 — the job runs successfully, but master is left without the regenerated file. Anyone running a clean upstream-branch canary that exercises the auto-gen path today will hit this.

## Recovery

I restored `azuredeploy.json` to the sample folder via a direct push to master, using the (correct) artifact from run 25822348426. The PR's expected outcome (regenerated JSON on master) is now in place; the issue is purely that it took a manual step instead of the on-merge auto-commit.

## Context

Original canary: #14764 (merged at `e2742e81`). Successful `/validate` run: `https://github.com/Azure/azure-quickstart-templates/actions/runs/25822348426`. Failing on-merge run: `https://github.com/Azure/azure-quickstart-templates/actions/runs/25822588613`.

Related: #14754, #14753.

cc @ouldsid

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.