elastic / elastic/integrations

[bug-hunter] validate-package-docs workflow reports success when docs-builder fails

Open
#18,099 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Handlebars
Stars
333
Forks
647
Avg merge
2d 17h
Merged PRs (30d)
225

Description

## Impact
`Validate Package Docs` can pass as a green required check even when docs validation fails, so broken package docs can merge. The workflow only emits an artifact for notification and never propagates a failing exit status.

## Reproduction Steps
1. Inspect the workflow logic:
- `.github/workflows/validate-package-docs.yml:43` sets `continue-on-error: true` on the docs validation step.
- `.github/workflows/validate-package-docs.yml:83-87` handles docs-builder failure by writing `validate-result/pr_number` only.
2. Run this minimal reproduction script (mirrors the failure-handling logic):

```bash
#!/usr/bin/env bash
set -euo pipefail
PR_NUMBER=123
mkdir -p "$PWD/validate-result"
if ! false; then
echo "\$\{PR_NUMBER}" > "$PWD/validate-result/pr_number"
fi
```

3. Execute it:

```bash
./repro.sh; code=$?; echo "exit_code=$code" && cat validate-result/pr_number
```

## Expected vs Actual
**Expected:** when docs-builder fails, the validation workflow fails (non-zero status), so branch protection can block merge.

**Actual:** failure is converted into a side-channel artifact, while the step/job can still succeed. Local reproduction output:

```text
exit_code=0
123
```

## Failing Test
```bash
#!/usr/bin/env bash
set -euo pipefail
PR_NUMBER=123
mkdir -p "$PWD/validate-result"
if ! false; then
echo "\$\{PR_NUMBER}" > "$PWD/validate-result/pr_number"
fi
```

This demonstrates the core bug: failed validation path still exits successfully.

## Evidence
- `.github/workflows/validate-package-docs.yml:43`
- `continue-on-error: true`
- `.github/workflows/validate-package-docs.yml:83-87`
- On docs-builder failure, workflow writes artifact marker only.
- `.github/workflows/validate-package-docs.yml:89-96`
- Uploads artifact, but no `exit 1` anywhere after validation failure.
- Introduced by commit `17d152f707` (`[ci] Inline notification into validation workflow, remove separate workflow`).

> [!NOTE]
>
> 🔒 Integrity filtering filtered 31 items
>
> Integrity filtering activated and filtered the following items during workflow execution.
> This happens when a tool call accesses a resource that does not meet the required integrity or secrecy level of the workflow.
>
> - issue:elastic/integrations#18095 (`list_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
> - issue:elastic/integrations#18094 (`list_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
> - issue:elastic/integrations#18093 (`list_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
> - issue:elastic/integrations#18090 (`list_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
> - issue:elastic/integrations#18076 (`list_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
> - issue:elastic/integrations#18068 (`list_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
> - issue:elastic/integrations#18067 (`list_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
> - issue:elastic/integrations#18066 (`list_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
> - issue:elastic/integrations#18065 (`list_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
> - issue:elastic/integrations#18064 (`list_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
> - issue:elastic/integrations#18062 (`list_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
> - issue:elastic/integrations#18056 (`list_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
> - issue:elastic/integrations#18054 (`list_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
> - issue:elastic/integrations#18035 (`list_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
> - issue:elastic/integrations#18030 (`list_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
> - issue:elastic/integrations#18014 (`list_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
> - ... and 15 more items
>
>

---
[What is this?](https://ela.st/github-ai-tools) | [From workflow: Bug Hunter](https://github.com/elastic/integrations/actions/runs/23643727391)

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.
> - [x] expires on Apr 3, 2026, 11:54 AM UTC

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.