elastic / elastic/integrations

[bug-hunter] Changelog skip-label check matches substrings and bypasses validation

Open
#19,381 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Handlebars
Stars
333
Forks
647
Avg merge
3d 4h
Merged PRs (30d)
209

Description

## Impact
PRs can bypass changelog link validation without the real skip label by using any label that merely contains `changelog-link-check:skip` as a substring (for example `changelog-link-check:skipper`). This creates false-pass CI behavior for the changelog link check.

## Reproduction Steps
1. Create and run this new repro script:

```bash
cat >/tmp/gh-aw/agent/repro_changelog_skip_label.sh <<'EOF'
#!/usr/bin/env bash
set -euo pipefail

SCRIPT="/home/runner/work/integrations/integrations/.buildkite/scripts/check_changelog_entries.sh"
source "$SCRIPT"

run_case() {
local labels="$1"
local expected="$2" # 0 means should skip, 1 means should not skip
if should_skip_changelog_check "$labels"; then
actual=0
else
actual=1
fi

printf 'labels=%q expected=%s actual=%s\n' "$labels" "$expected" "$actual"
if [[ "$actual" != "$expected" ]]; then
echo "FAIL"
return 1
fi
}

# Control: exact label should skip.
run_case "team:obs,changelog-link-check:skip,docs" 0

# Bug: substring label should NOT skip, but currently does.
run_case "team:obs,changelog-link-check:skipper,docs" 1

echo "PASS"
EOF
chmod +x /tmp/gh-aw/agent/repro_changelog_skip_label.sh
/tmp/gh-aw/agent/repro_changelog_skip_label.sh
```

## Expected vs Actual
**Expected:** only an exact `changelog-link-check:skip` label should skip the check.

**Actual:** substring matches are treated as skip:

```text
labels=team:obs\,changelog-link-check:skip\,docs expected=0 actual=0
labels=team:obs\,changelog-link-check:skipper\,docs expected=1 actual=0
FAIL
```

## Failing Test
```bash
#!/usr/bin/env bash
set -euo pipefail

SCRIPT="/home/runner/work/integrations/integrations/.buildkite/scripts/check_changelog_entries.sh"
source "$SCRIPT"

run_case() {
local labels="$1"
local expected="$2" # 0 means should skip, 1 means should not skip
if should_skip_changelog_check "$labels"; then
actual=0
else
actual=1
fi

printf 'labels=%q expected=%s actual=%s\n' "$labels" "$expected" "$actual"
if [[ "$actual" != "$expected" ]]; then
echo "FAIL"
return 1
fi
}

run_case "team:obs,changelog-link-check:skip,docs" 0
run_case "team:obs,changelog-link-check:skipper,docs" 1

echo "PASS"
```

## Evidence
- Substring skip check in script: `.buildkite/scripts/check_changelog_entries.sh:71`
- `[[ "\$\{labels}" == *"\$\{CHANGELOG_SKIP_LABEL}"* ]]`
- Substring regex in pipeline condition: `.buildkite/pipeline.yml:110`
- `!(build.env("GITHUB_PR_LABELS") =~ /\$\{CHANGELOG_SKIP_LABEL}/)`
- No matching existing open issue found for this specific substring-bypass behavior; related open issue `#19360` is about quoted changelog links, not skip-label matching.

> [!NOTE]
>
> 🔒 Integrity filter blocked 2 items
>
> The following items were blocked because they don't meet the GitHub integrity level.
>
> - [#18539](https://github.com/elastic/integrations/issues/18539) `search_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
> - [#17372](https://github.com/elastic/integrations/pull/17372) `search_pull_requests`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
>
> To allow these resources, lower `min-integrity` in your GitHub frontmatter:
>
> ```yaml
> tools:
> github:
> min-integrity: approved # merged | approved | unapproved | none
> ```
>
>

---
[What is this?](https://ela.st/github-ai-tools) | [From workflow: Bug Hunter](https://github.com/elastic/integrations/actions/runs/26950373700)

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.
> - [x] expires on Jun 11, 2026, 12:12 PM UTC

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.