elastic / elastic/ai-github-actions
[bug-hunter] estc-pr-buildkite-detective docs default disagrees with workflow behavior
- Dominant language
- Python
- Stars
- 11
- Forks
- 16
- Avg merge
- 22h 9m
- Merged PRs (30d)
- 31
Description
## Impact
Users configuring `estc-pr-buildkite-detective` from the published docs/example are told `report-failure-as-issue` defaults to `true`, but the actual workflow default is `false`. In practice, failed Buildkite detective runs do **not** auto-create failure issues unless callers explicitly override the input, reducing failure visibility and breaking expected triage behavior.
## Reproduction Steps
1. Save and run this script from the repository root:
```python
#!/usr/bin/env python3
import pathlib, re, sys
root = pathlib.Path('/home/runner/work/ai-github-actions/ai-github-actions')
wf = (root / '.github/workflows/gh-aw-estc-pr-buildkite-detective.md').read_text(encoding='utf-8')
readme = (root / 'gh-agent-workflows/estc-pr-buildkite-detective/README.md').read_text(encoding='utf-8')
docs = (root / 'docs/workflows/gh-agent-workflows/estc-pr-buildkite-detective.md').read_text(encoding='utf-8')
src_m = re.search(r"report-failure-as-issue:\\n(?:.*\\n){0,6}?\\s*default:\\s*(true|false)", wf)
readme_m = re.search(r"\\| `report-failure-as-issue` \\|[^\\n]*\\| `?(true|false)`? \\|", readme)
docs_m = re.search(r"\\| `report-failure-as-issue` \\|[^\\n]*\\| `?(true|false)`? \\|", docs)
if not (src_m and readme_m and docs_m):
print('FAILED: could not parse one or more defaults')
sys.exit(2)
src = src_m.group(1)
rd = readme_m.group(1)
dc = docs_m.group(1)
print(f'workflow_default={src}')
print(f'readme_default={rd}')
print(f'docs_default={dc}')
if len({src, rd, dc}) != 1:
print('BUG: documented defaults disagree with workflow default')
sys.exit(1)
print('OK: defaults are consistent')
```
2. Run:
```bash
python /tmp/gh-aw/agent/repro-estc-buildkite-default-mismatch.py
```
## Expected vs Actual
**Expected:** Documentation and workflow source agree on the default for `report-failure-as-issue`.
**Actual:** Source default is `false`, while both docs locations say `true`.
Observed output:
```text
workflow_default=false
readme_default=true
docs_default=true
BUG: documented defaults disagree with workflow default
```
## Failing Test
```python
#!/usr/bin/env python3
import pathlib, re, sys
root = pathlib.Path('/home/runner/work/ai-github-actions/ai-github-actions')
wf = (root / '.github/workflows/gh-aw-estc-pr-buildkite-detective.md').read_text(encoding='utf-8')
readme = (root / 'gh-agent-workflows/estc-pr-buildkite-detective/README.md').read_text(encoding='utf-8')
docs = (root / 'docs/workflows/gh-agent-workflows/estc-pr-buildkite-detective.md').read_text(encoding='utf-8')
src_m = re.search(r"report-failure-as-issue:\\n(?:.*\\n){0,6}?\\s*default:\\s*(true|false)", wf)
readme_m = re.search(r"\\| `report-failure-as-issue` \\|[^\\n]*\\| `?(true|false)`? \\|", readme)
docs_m = re.search(r"\\| `report-failure-as-issue` \\|[^\\n]*\\| `?(true|false)`? \\|", docs)
if not (src_m and readme_m and docs_m):
raise SystemExit(2)
src = src_m.group(1)
rd = readme_m.group(1)
dc = docs_m.group(1)
print(src, rd, dc)
raise SystemExit(0 if len({src, rd, dc}) == 1 else 1)
```
## Evidence
- `.github/workflows/gh-aw-estc-pr-buildkite-detective.md` (input definition): `report-failure-as-issue` default is `false`.
- `gh-agent-workflows/estc-pr-buildkite-detective/README.md` (inputs table): default documented as `true`.
- `docs/workflows/gh-agent-workflows/estc-pr-buildkite-detective.md` (inputs table): default documented as `true`.
- `gh-agent-workflows/estc-pr-buildkite-detective/example.yml` does not pass `report-failure-as-issue`, so runtime behavior uses the source default (`false`).
---
[What is this?](https://ela.st/github-ai-tools) | [From workflow: Trigger Bug Hunter](https://github.com/elastic/ai-github-actions/actions/runs/34593221319)
Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.
> - [x] expires on Sep 18, 2026, 11:27 AM UTC
Contributor guide
Research direction
Read the input definition in .github/workflows/gh-aw-estc-pr-buildkite-detective.md and compare it with the inputs tables in gh-agent-workflows/estc-pr-buildkite-detective/README.md and docs/workflows/gh-agent-workflows/estc-pr-buildkite-detective.md. Run the provided Python consistency check from the repository root. Done means all three reported defaults agree and the check exits successfully.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- github-actions, python
- Domain
- ci-cd, documentation
- Issue type
- Documentation
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 90/100