dotnet / dotnet/runtime

[ci-scan-feedback] Add build-break dedup search variation to KBE authoring

Open
#132,669 1 comment 0 reactions 0 assignees View on GitHub
agentic-workflows area-Infrastructure untriaged
Dominant language
C#
Stars
18.3k
Forks
5.6k
PR merge metrics
PR metrics pending

Description

## Triggering signals

- Scanner re-filed the same **`PALTests Send-to-Helix SYSTEM_ACCESSTOKEN variable not found`** build break as **5 separate open KBEs** over 3 days: [#132545](https://github.com/dotnet/runtime/issues/132545), [#132560](https://github.com/dotnet/runtime/issues/132560), [#132614](https://github.com/dotnet/runtime/issues/132614), [#132648](https://github.com/dotnet/runtime/issues/132648), [#132656](https://github.com/dotnet/runtime/issues/132656). Titles drift across runs ("Send to Helix" / "Send job to Helix" / "Send-to-Helix"), defeating `in:title` prose matching.
- Same pattern for the Mono AOT recursive-generic break: [#132559](https://github.com/dotnet/runtime/issues/132559) and [#132647](https://github.com/dotnet/runtime/issues/132647) both file `JIT.Generics_ro` AOT-compile failure under differently-worded titles.
- Scanner 30d closure cohort: **10 of 34 closures wrong (29%)**, 9 of them `duplicate` (#131368, #131418, #131742, #131743, #131744, #131980, #131998, #132392, #132516). This is a dedup failure, not triage noise — build-break signatures have no test name or assertion, so existing "Search for an existing KBE" variations 1–7 (all test-name/assertion-keyed) never fire.

## Proposed edits

- `.github/workflows/shared/create-kbe.instructions.md` (`## Search for an existing KBE`, new variation 8 after the bare-signature variation 7): add a build-break-specific dedup search keyed on the two stable tokens — the invariant error phrase (e.g. `SYSTEM_ACCESSTOKEN`, AOT'd assembly `JIT.Generics_ro`) and the `Build error leg` descriptor — searched `in:body` rather than relying on drifting title prose. Rationale: build breaks are exactly the signatures variations 1–7 miss.
- Same file (summary paragraph after the variations): one sentence documenting what variation 8 catches, matching the existing per-variation summary style.

## Expected behavior change

The next scanner run will recognize a build break already filed under a differently-worded title as an `existing-kbe` hit (matching on error phrase + leg descriptor `in:body`) and record it instead of filing a fresh KBE — eliminating the `SYSTEM_ACCESSTOKEN`-style multi-day re-file storms that drove the 30d wrong-closure rate to 29%.

> [!NOTE]
> This pull request was generated by an AI agent (GitHub Copilot).

> Generated by [CI Outer-Loop Failure Scanner — Feedback](https://github.com/dotnet/runtime/actions/runs/32641342962) · opus48 · 539.6 AIC · ⌖ 25.3 AIC · ⊞ 20.6K · [◷](https://github.com/search?q=repo%3Adotnet%2Fruntime+%22gh-aw-workflow-id%3A+ci-failure-scan-feedback%22&type=pullrequests)

---

> [!NOTE]
> This was originally intended as a pull request, but the git push operation failed.
>
> **Original error:** The process '/usr/bin/git' failed with exit code 1
>
> **Workflow Run:** [View run details and download bundle artifact](https://github.com/dotnet/runtime/actions/runs/32641342962)
>
> The bundle file is available in the `agent` artifact in the workflow run linked above.

To create a pull request with the changes:

```sh
# Download the artifact from the workflow run
gh run download 32641342962 -n agent -D /tmp/agent-32641342962

# Fetch the bundle into a temporary ref, then update the local branch
git fetch /tmp/agent-32641342962/aw-ci-scan-feedback-build-break-dedup.bundle refs/heads/ci-scan-feedback/build-break-dedup:refs/bundles/create-pr-ci-scan-feedback-build-break-dedup-88148e1bf6019fd2-5bc7bb8f
git update-ref refs/heads/ci-scan-feedback/build-break-dedup-88148e1bf6019fd2 refs/bundles/create-pr-ci-scan-feedback-build-break-dedup-88148e1bf6019fd2-5bc7bb8f
git checkout ci-scan-feedback/build-break-dedup-88148e1bf6019fd2
# Ensure the working tree matches the updated branch
git reset --hard
# Remove the temporary bundle ref
git update-ref -d refs/bundles/create-pr-ci-scan-feedback-build-break-dedup-88148e1bf6019fd2-5bc7bb8f

# Push the branch to origin
git push origin ci-scan-feedback/build-break-dedup-88148e1bf6019fd2

# Create the pull request
gh pr create --title '[ci-scan-feedback] Add build-break dedup search variation to KBE authoring' --base main --head ci-scan-feedback/build-break-dedup-88148e1bf6019fd2 --repo dotnet/runtime
```

Contributor guide

Open the contributing guide

Research direction

Read .github/workflows/shared/create-kbe.instructions.md, especially the existing search variations and their summary paragraph. Add the specified build-break variation and matching summary sentence, then verify the instructions require the invariant error phrase and Build error leg descriptor to be searched in the body; the next scanner run should avoid duplicate KBEs.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions
Domain
ci-cd, documentation
Issue type
Documentation
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.