[ci-scan-feedback] Reject shared-console-log suite-harness KBE signatures
- Dominant language
- C#
- Stars
- 18.3k
- Forks
- 5.6k
- PR merge metrics
- PR metrics pending
Description
## Triggering signals
- KBE #131449 — MEMBER janvorli (2026-08-07): *"This issue seems to be a kitchen sink for all kinds of profiler tests failures."* https://github.com/dotnet/runtime/issues/131449#issuecomment
- KBE #131449 — MEMBER noahfalk (2026-08-22): re-scoped the KBE so it "only matches nongcheap.cmd. Previously the KBE matched the Helix console log which runs many tests. Whenever some other profiler test failed the [signature matched]." https://github.com/dotnet/runtime/issues/131449
- Rubric finding: the array signature paired a per-invocation `Running test: ...nongcheap.cmd` announcement with a suite-wide harness error (`PROFILER TEST PASSES` missing); both co-exist in the profiler suite's single shared Helix console log, so the KBE matched whenever *any* `profiler/**` test failed. A prior tracker snapshot claimed this class was addressed, but no `[ci-scan-feedback]` PR actually landed the rule.
## Proposed edits
- `.github/workflows/shared/create-kbe.instructions.md` (Signature specificity → reject list, ~line 530): add a reject bullet for signatures pairing a per-invocation announcement line with a suite-wide harness error that share one console log — tied to the #131449 kitchen-sink signal.
- `.github/workflows/shared/create-kbe.instructions.md` (Bad vs good signatures table, ~line 556): add a Bad/Good row using the concrete `gcheapenumeration` profiler case, showing the anchor should be the failing test's own error output from a per-test log source rather than the harness aggregate.
## Expected behavior change
The next scanner run will refuse to build a KBE signature from a per-invocation announcement line combined with a shared-harness pass/fail summary when both live in one suite-wide console log, avoiding the over-broad "kitchen sink" KBE that matches every unrelated failure in the same test suite.
> [!NOTE]
> This PR description was generated by an AI/Copilot agentic workflow.
> Generated by [CI Outer-Loop Failure Scanner — Feedback](https://github.com/dotnet/runtime/actions/runs/33632772611) · opus48 · 369.3 AIC · ⌖ 25.3 AIC · ⊞ 20.6K · [◷](https://github.com/search?q=repo%3Adotnet%2Fruntime+%22gh-aw-workflow-id%3A+ci-failure-scan-feedback%22&type=pullrequests)
---
> [!NOTE]
> This was originally intended as a pull request, but the git push operation failed.
>
> **Original error:** The process '/usr/bin/git' failed with exit code 1
>
> **Workflow Run:** [View run details and download bundle artifact](https://github.com/dotnet/runtime/actions/runs/33632772611)
>
> The bundle file is available in the `agent` artifact in the workflow run linked above.
To create a pull request with the changes:
```sh
# Download the artifact from the workflow run
gh run download 33632772611 -n agent -D /tmp/agent-33632772611
# Fetch the bundle into a temporary ref, then update the local branch
git fetch /tmp/agent-33632772611/aw-ci-scan-feedback-shared-console-log-signature.bundle refs/heads/ci-scan-feedback-shared-console-log-signature:refs/bundles/create-pr-ci-scan-feedback-shared-console-log-signature-ecfc2f9d38a585eb-d685b77d
git update-ref refs/heads/ci-scan-feedback-shared-console-log-signature-ecfc2f9d38a585eb refs/bundles/create-pr-ci-scan-feedback-shared-console-log-signature-ecfc2f9d38a585eb-d685b77d
git checkout ci-scan-feedback-shared-console-log-signature-ecfc2f9d38a585eb
# Ensure the working tree matches the updated branch
git reset --hard
# Remove the temporary bundle ref
git update-ref -d refs/bundles/create-pr-ci-scan-feedback-shared-console-log-signature-ecfc2f9d38a585eb-d685b77d
# Push the branch to origin
git push origin ci-scan-feedback-shared-console-log-signature-ecfc2f9d38a585eb
# Create the pull request
gh pr create --title '[ci-scan-feedback] Reject shared-console-log suite-harness KBE signatures' --base main --head ci-scan-feedback-shared-console-log-signature-ecfc2f9d38a585eb --repo dotnet/runtime
```
Contributor guide
Research direction
Open .github/workflows/shared/create-kbe.instructions.md and read the reject list around line 530 and the Bad vs good signatures table around line 556. Add the shared-console-log rule and the gcheapenumeration example described in the issue. Done means the instructions reject per-invocation announcements paired with suite-wide harness results and show a per-test error as the acceptable anchor.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- github-actions
- Domain
- ci-cd, documentation
- Issue type
- Documentation
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 76/100