elastic / elastic/integrations

[bug-hunter] PR Buildkite Detective runs on check_run events without associated pull requests

Open
#18,217 0 comments 0 reactions 0 assignees View on GitHub
stale
Dominant language
Handlebars
Stars
333
Forks
647
Avg merge
3d 4h
Merged PRs (30d)
209

Description

## Impact
The `PR Buildkite Detective` workflow can run for failed Buildkite `check_run` events that are not associated with any pull request. This triggers PR-specific automation in non-PR contexts, creating noisy/incorrect workflow executions.

## Reproduction Steps
1. Inspect the workflow condition in `.github/workflows/pr-buildkite-detective.yml`:
- Line 17: `toJSON(github.event.check_run.pull_requests) != '[]'`
2. Run this reproduction script from repository root:

```python
import json
from pathlib import Path

text = Path('.github/workflows/pr-buildkite-detective.yml').read_text()
line = [l.rstrip() for l in text.splitlines() if 'toJSON(github.event.check_run.pull_requests)' in l][0]

payload = {
'check_run': {
'conclusion': 'failure',
'app': {'slug': 'buildkite'},
'pull_requests': None,
}
}

expr_result = (
payload['check_run']['conclusion'] == 'failure'
and payload['check_run']['app']['slug'] == 'buildkite'
and json.dumps(payload['check_run']['pull_requests']) != '[]'
)

print('condition_line:', line)
print('toJSON(pull_requests)=', json.dumps(payload['check_run']['pull_requests']))
print('guard_result=', expr_result)

if expr_result:
print('FAIL: guard passes even though no PR is associated')
raise SystemExit(1)
```

## Expected vs Actual
**Expected:** The job should not run when `github.event.check_run.pull_requests` is `null`/missing (no associated PR).

**Actual:** The guard evaluates `true` because `toJSON(null)` is `"null"`, which is not equal to `'[]'`.

Observed output:
- `toJSON(pull_requests)= null`
- `guard_result= True`
- `FAIL: guard passes even though no PR is associated`

## Failing Test
```python
import json
from pathlib import Path

text = Path('.github/workflows/pr-buildkite-detective.yml').read_text()
line = [l.rstrip() for l in text.splitlines() if 'toJSON(github.event.check_run.pull_requests)' in l][0]

payload = {
'check_run': {
'conclusion': 'failure',
'app': {'slug': 'buildkite'},
'pull_requests': None,
}
}

expr_result = (
payload['check_run']['conclusion'] == 'failure'
and payload['check_run']['app']['slug'] == 'buildkite'
and json.dumps(payload['check_run']['pull_requests']) != '[]'
)

print('condition_line:', line)
print('toJSON(pull_requests)=', json.dumps(payload['check_run']['pull_requests']))
print('guard_result=', expr_result)

if expr_result:
print('FAIL: guard passes even though no PR is associated')
raise SystemExit(1)
```

## Evidence
- Workflow file: `.github/workflows/pr-buildkite-detective.yml:15-17`
- Current condition:
- `github.event.check_run.conclusion == 'failure'`
- `github.event.check_run.app.slug == 'buildkite'`
- `toJSON(github.event.check_run.pull_requests) != '[]'`
- Deterministic local reproduction shows `null` passes this guard unexpectedly.

> [!NOTE]
>
> 🔒 Integrity filtering filtered 3 items
>
> Integrity filtering activated and filtered the following items during workflow execution.
> This happens when a tool call accesses a resource that does not meet the required integrity or secrecy level of the workflow.
>
> - issue:#unknown (`search_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
> - [#17972](https://github.com/elastic/integrations/issues/17972) (`search_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
> - [#18153](https://github.com/elastic/integrations/issues/18153) (`search_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
>
>

---
[What is this?](https://ela.st/github-ai-tools) | [From workflow: Bug Hunter](https://github.com/elastic/integrations/actions/runs/23944353873)

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.
> - [x] expires on Apr 10, 2026, 11:33 AM UTC

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.