[FEA]: pathfinder "all_must_work" alert mechanism
@rwgk is already working on this.
Since Apr 17, 2026.
- Dominant language
- Cython
- Stars
- 3.4k
- Forks
- 329
- Avg merge
- 1d 23h
- Merged PRs (30d)
- 116
Description
Problem
The pathfinder all_must_work tests can fail in CI due to environment differences (e.g., MCDM driver mode, missing DLLs, package version mismatches) that are difficult to reproduce locally. Currently, these failures block the entire CI workflow.
We want to surface pathfinder issues without blocking PRs, while ensuring they don't go unnoticed.
Motivation
Adding a dedicated .github/workflows/pathfinder.yml with its own matrix would:
- Require significant additional CI code
- Add long-term maintenance burden
- Duplicate matrix definitions across workflows
We need a lighter-weight alternative that still provides visibility into pathfinder health.
Proposed Solution
Use a "sentinel issue" pattern: a single GitHub issue that tracks pathfinder all_must_work failures over time.
Behavior
- Create a tracking issue: "pathfinder all_must_work alert"
- When
all_must_workfails in CI:- Post a comment to the issue with job details (run link, commit, PR, etc.)
- Reopen the issue if it's currently closed
- The test step uses
continue-on-error: trueso it doesn't block the workflow
Benefits
- Single source of truth - all failures tracked in one place with history
- Automatic notifications - (only) maintainers subscribed to the issue get alerts
- Low maintenance - no new workflow files or matrices to maintain
- Self-healing - closing the issue signals "all clear"; it auto-reopens on failure
- Actionable - issue can be assigned, labeled, added to projects
Implementation Sketch
- name: Pathfinder all_must_work
continue-on-error: true
id: pathfinder
run: run-tests pathfinder-strict
- name: Report pathfinder failure to tracking issue
if: always() && steps.pathfinder.outcome == 'failure'
env:
GH_TOKEN: ${{ github.token }}
run: |
ISSUE_TITLE="Pathfinder all_must_work alert"
# Find existing issue by title (searches both open and closed)
ISSUE_NUMBER=$(gh issue list --search "in:title \"$ISSUE_TITLE\"" --state all --json number --jq '.[0].number')
if [ -z "$ISSUE_NUMBER" ]; then
# Create the tracking issue if it doesn't exist
ISSUE_NUMBER=$(gh issue create \
--title "$ISSUE_TITLE" \
--body "Tracking issue for pathfinder all_must_work CI failures. This issue is automatically reopened when failures occur." \
--label "alert,pathfinder" \
--json number --jq '.number')
fi
# Post comment with failure details
gh issue comment "$ISSUE_NUMBER" --body "## ⚠️ Failure detected
| Field | Value |
|-------|-------|
| **Run** | ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} |
| **Job** | ${{ github.job }} |
| **Commit** | \`${{ github.sha }}\` |
| **Branch** | \`${{ github.ref_name }}\` |
| **PR** | ${{ github.event.pull_request.html_url || 'N/A' }} |
| **Actor** | @${{ github.actor }} |
"
# Reopen if currently closed
gh issue reopen "$ISSUE_NUMBER" 2>/dev/null || true
Optional: Recovery Notification
- name: Report pathfinder recovery
if: always() && steps.pathfinder.outcome == 'success'
env:
GH_TOKEN: ${{ github.token }}
run: |
# Only comment if issue is open (indicating recent failure)
ISSUE_NUMBER=$(gh issue list --search "in:title \"Pathfinder all_must_work alert\"" --state open --json number --jq '.[0].number')
if [ -n "$ISSUE_NUMBER" ]; then
gh issue comment "$ISSUE_NUMBER" --body "✅ Passing again as of ${{ github.sha }}"
fi
Alternatives Considered
| Approach | Pros | Cons |
|---|---|---|
| Dedicated workflow + matrix | Full control, separate status | High maintenance, duplicated config |
::warning:: annotations only |
Simple | Easy to miss, no history |
| PR labels | Filterable | No notifications, manual cleanup |
| PR comments | Visible per-PR | No aggregate view, noisy |
| Sentinel issue (proposed) | Central tracking, notifications, history | Comments accumulate |
Requirements
issues: writepermission in the workflow
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.