microsoft / microsoft/aspire

[automated] Fix main CI failures being filed as PR infrastructure issues

Open
#19,454 0 comments 1 reaction 2 assignees Claimed by @radical View on GitHub
area-engineering-systems
Dominant language
C#
Stars
6.3k
Forks
991
Avg merge
2d 15h
Merged PRs (30d)
196

Description

[automated]

### Is there an existing issue for this?

- [x] I have searched the existing issues

### Describe the bug

The `Analyze CI Failure` workflow can describe a failed `push` run on `main` as a PR-branch failure and classify deterministic repository breakage as infrastructure failure.

Issue #19149 is a concrete example. Its source CI run, https://github.com/microsoft/aspire/actions/runs/31203621605, had:

- Event: `push`
- Branch: `main`
- Failed SHA: `5e7a15c05d91b1b8257f4ed620981b4e93e390c7`
- Display title: `Mark ResourceUrlAnnotation.DisplayOrder as obsolete (#19090)`

The generated issue instead said `Pull request: #19090` and described a “stale PR branch.” PR #19090 was unrelated; it only triggered the next `main` build.

The actual failure was a semantic merge conflict between #19084 and #18976. PR #19084 added tests using `ForSubscription` and `ForTenant`; #18976 merged 27 seconds later and renamed those APIs to `CreateForSubscription` and `CreateForTenant`. Both PRs were green independently, but their combined state left `main` uncompilable.

The workflow treated the failure as `infra-failure` because it was unrelated to #19090's changed files. That classification caused #19149 to be opened with misleading PR attribution.

### Expected Behavior

The workflow should determine run scope from immutable workflow metadata before invoking the analysis agent:

- PR validation failures caused by PR changes should be reported on the PR without creating a CI-cause issue.
- Failed `main` pushes caused by repository changes should create or update an explicitly labeled main-CI-break issue.
- A merge PR associated with a `main` push should be described as the triggering merge, not presumed to be causal.
- Infrastructure and flaky failures should continue to reuse their stable cause records.

A generated issue for this example should have identified the affected branch as `main`, treated #19090 as non-causal context, and considered all merges since the last green main run.

### Steps To Reproduce

1. Inspect failed CI run https://github.com/microsoft/aspire/actions/runs/31203621605.
2. Observe that its event is `push` and its branch is `main`.
3. Inspect Analyze CI Failure run https://github.com/microsoft/aspire/actions/runs/31207524902.
4. Inspect generated issue #19149.
5. Compare the issue's PR-branch wording and `infra-failure` classification with the actual main-branch semantic merge conflict fixed by #19148.

### Exceptions (if any)

```text
error CS0117: 'AzureBicepResourceScope' does not contain a definition for 'ForSubscription'
error CS0117: 'AzureBicepResourceScope' does not contain a definition for 'ForTenant'
```

### Aspire doctor output

N/A — this is a GitHub Actions workflow bug.

### Anything else?

Suggested workflow changes:

1. Emit a deterministic `run_scope` (`main` or `pull-request`) from `event` and `head_branch` before agent analysis.
2. For failed main runs, record the last successful main SHA and candidate merges between that SHA and the failed SHA.
3. Rename PR metadata for main runs to `triggering_merge_pr` and state explicitly that it is not necessarily causal.
4. Add a `main-repository-breakage` classification instead of forcing deterministic code failures unrelated to the triggering PR into `infra-failure`.
5. Use distinct issue templates and titles, such as `[Main CI Failure] ...`, for main repository breaks.
6. Add regression coverage ensuring a `push`/`main` run cannot be rendered as a PR-branch failure and cannot classify compile errors as infrastructure failures.

PR #19148 fixed the underlying compile break, and the subsequent main CI run https://github.com/microsoft/aspire/actions/runs/31211923676 succeeded. Issue #19149 remains open because the fix PR did not link it correctly.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.