dotnet / dotnet/runtime

[ci-scan-feedback] Reject shared-console-log suite-harness KBE signatures

Open Beginner friendly
#133,080 1 comment 0 reactions 0 assignees View on GitHub
agentic-workflows area-Infrastructure untriaged
Dominant language
C#
Stars
18.3k
Forks
5.6k
PR merge metrics
PR metrics pending

Description

## Triggering signals

- KBE #131449 — MEMBER janvorli (2026-08-07): *"This issue seems to be a kitchen sink for all kinds of profiler tests failures."* https://github.com/dotnet/runtime/issues/131449#issuecomment
- KBE #131449 — MEMBER noahfalk (2026-08-22): re-scoped the KBE so it "only matches nongcheap.cmd. Previously the KBE matched the Helix console log which runs many tests. Whenever some other profiler test failed the [signature matched]." https://github.com/dotnet/runtime/issues/131449
- Rubric finding: the array signature paired a per-invocation `Running test: ...nongcheap.cmd` announcement with a suite-wide harness error (`PROFILER TEST PASSES` missing); both co-exist in the profiler suite's single shared Helix console log, so the KBE matched whenever *any* `profiler/**` test failed. A prior tracker snapshot claimed this class was addressed, but no `[ci-scan-feedback]` PR actually landed the rule.

## Proposed edits

- `.github/workflows/shared/create-kbe.instructions.md` (Signature specificity → reject list, ~line 530): add a reject bullet for signatures pairing a per-invocation announcement line with a suite-wide harness error that share one console log — tied to the #131449 kitchen-sink signal.
- `.github/workflows/shared/create-kbe.instructions.md` (Bad vs good signatures table, ~line 556): add a Bad/Good row using the concrete `gcheapenumeration` profiler case, showing the anchor should be the failing test's own error output from a per-test log source rather than the harness aggregate.

## Expected behavior change

The next scanner run will refuse to build a KBE signature from a per-invocation announcement line combined with a shared-harness pass/fail summary when both live in one suite-wide console log, avoiding the over-broad "kitchen sink" KBE that matches every unrelated failure in the same test suite.

> [!NOTE]
> This PR description was generated by an AI/Copilot agentic workflow.

> Generated by [CI Outer-Loop Failure Scanner — Feedback](https://github.com/dotnet/runtime/actions/runs/33632772611) · opus48 · 369.3 AIC · ⌖ 25.3 AIC · ⊞ 20.6K · [◷](https://github.com/search?q=repo%3Adotnet%2Fruntime+%22gh-aw-workflow-id%3A+ci-failure-scan-feedback%22&type=pullrequests)

---

> [!NOTE]
> This was originally intended as a pull request, but the git push operation failed.
>
> **Original error:** The process '/usr/bin/git' failed with exit code 1
>
> **Workflow Run:** [View run details and download bundle artifact](https://github.com/dotnet/runtime/actions/runs/33632772611)
>
> The bundle file is available in the `agent` artifact in the workflow run linked above.

To create a pull request with the changes:

```sh
# Download the artifact from the workflow run
gh run download 33632772611 -n agent -D /tmp/agent-33632772611

# Fetch the bundle into a temporary ref, then update the local branch
git fetch /tmp/agent-33632772611/aw-ci-scan-feedback-shared-console-log-signature.bundle refs/heads/ci-scan-feedback-shared-console-log-signature:refs/bundles/create-pr-ci-scan-feedback-shared-console-log-signature-ecfc2f9d38a585eb-d685b77d
git update-ref refs/heads/ci-scan-feedback-shared-console-log-signature-ecfc2f9d38a585eb refs/bundles/create-pr-ci-scan-feedback-shared-console-log-signature-ecfc2f9d38a585eb-d685b77d
git checkout ci-scan-feedback-shared-console-log-signature-ecfc2f9d38a585eb
# Ensure the working tree matches the updated branch
git reset --hard
# Remove the temporary bundle ref
git update-ref -d refs/bundles/create-pr-ci-scan-feedback-shared-console-log-signature-ecfc2f9d38a585eb-d685b77d

# Push the branch to origin
git push origin ci-scan-feedback-shared-console-log-signature-ecfc2f9d38a585eb

# Create the pull request
gh pr create --title '[ci-scan-feedback] Reject shared-console-log suite-harness KBE signatures' --base main --head ci-scan-feedback-shared-console-log-signature-ecfc2f9d38a585eb --repo dotnet/runtime
```

Contributor guide

Open the contributing guide

Research direction

Open .github/workflows/shared/create-kbe.instructions.md and read the reject list around line 530 and the Bad vs good signatures table around line 556. Add the shared-console-log rule and the gcheapenumeration example described in the issue. Done means the instructions reject per-invocation announcements paired with suite-wide harness results and show a per-test error as the acceptable anchor.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions
Domain
ci-cd, documentation
Issue type
Documentation
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
76/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.