elastic / elastic/integrations

[bug-hunter] validate-yaml-dashboards skips validation for JSON-only dashboard edits

Open
#18,310 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Handlebars
Stars
333
Forks
647
Avg merge
2d 17h
Merged PRs (30d)
225

Description

## Impact
Pull requests that modify `packages/**/kibana/dashboard/*.json` can pass the `Validate Dashboard Compilation` workflow without any lint/upgrade/compile validation. This allows stale or invalid committed dashboard JSON to merge undetected.

## Reproduction Steps
1. Create a temporary worktree and commit a JSON-only dashboard change:
```bash
git worktree add /tmp/gh-aw/agent/repro-json-skip HEAD
cd /tmp/gh-aw/agent/repro-json-skip
git switch -c repro-json-skip
json_file=$(find packages -path '*/kibana/dashboard/*.json' | sed -n '1p')
printf '\n' >> "$json_file"
git add "$json_file"
git -c user.name='repro' -c user.email='repro@example.com' commit -m 'repro: json-only dashboard change'
```
2. Run the same path checks used by the workflow:
```bash
git diff --name-only HEAD~1...HEAD -- 'packages/**/kibana/dashboard/*.json'

git diff --name-only HEAD~1...HEAD -- \
'packages/**/_dev/shared/**/*.yaml' \
'packages/**/_dev/shared/kibana/**/*.yaml'
```

## Expected vs Actual
**Expected:** A JSON dashboard change should be picked up for validation by this workflow (or the workflow should not trigger on JSON changes).

**Actual:** JSON change is detected by trigger glob, but the `find-changed` step only checks YAML globs and returns empty, so subsequent validation steps are skipped.

Observed output from local reproduction:
```text
--- changed files matching workflow trigger JSON path ---
packages/microsoft_sentinel/kibana/dashboard/microsoft_sentinel-c0a331fd-c7d9-452e-b5db-0f062479c779.json
--- changed files matching workflow find-changed YAML globs ---
(empty)
RESULT: find-changed step treats this PR as having no changed YAML dashboard files and exits 0.
```

## Failing Test
```bash
#!/usr/bin/env bash
set -euo pipefail

repo=/home/runner/work/integrations/integrations
repro_dir=/tmp/gh-aw/agent/repro-json-skip

cd "$repo"
git branch -D repro-json-skip >/dev/null 2>&1 || true
git worktree remove --force "$repro_dir" >/dev/null 2>&1 || true
git worktree prune

git worktree add --quiet "$repro_dir" HEAD
cd "$repro_dir"
git switch --quiet -c repro-json-skip

json_file=$(find packages -path '*/kibana/dashboard/*.json' | sed -n '1p')
printf '\n' >> "$json_file"
git add "$json_file"
git -c user.name='repro' -c user.email='repro@example.com' commit -q -m 'repro: json-only dashboard change'

json_changed=$(git diff --name-only HEAD~1...HEAD -- 'packages/**/kibana/dashboard/*.json')
yaml_changed=$(git diff --name-only HEAD~1...HEAD -- \
'packages/**/_dev/shared/**/*.yaml' \
'packages/**/_dev/shared/kibana/**/*.yaml' \
2>/dev/null || true)

if [[ -n "$json_changed" && -z "$yaml_changed" ]]; then
echo "BUG REPRODUCED: workflow trigger sees JSON change but find-changed sees no files"
exit 1
fi
```

## Evidence
- Workflow trigger includes JSON dashboard changes:
- `.github/workflows/validate-yaml-dashboards.yml:8-11`
- `find-changed` step only diffs YAML globs, then exits early when empty:
- `.github/workflows/validate-yaml-dashboards.yml:39-49`
- All validation stages are gated on `find-changed` outputs from YAML-only detection:
- `.github/workflows/validate-yaml-dashboards.yml:69`, `121`, `157`

> [!NOTE]
>
> 🔒 Integrity filtering filtered 22 items
>
> Integrity filtering activated and filtered the following items during workflow execution.
> This happens when a tool call accesses a resource that does not meet the required integrity or secrecy level of the workflow.
>
> - issue:elastic/integrations#unknown (`search_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
> - [#17815](https://github.com/elastic/integrations/issues/17815) (`search_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
> - issue:elastic/integrations#17851 (`list_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
> - issue:elastic/integrations#18307 (`list_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
> - issue:elastic/integrations#18306 (`list_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
> - issue:elastic/integrations#17956 (`list_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
> - issue:elastic/integrations#17955 (`list_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
> - issue:elastic/integrations#17954 (`list_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
> - issue:elastic/integrations#17958 (`list_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
> - issue:elastic/integrations#17957 (`list_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
> - issue:elastic/integrations#17716 (`list_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
> - issue:elastic/integrations#17692 (`list_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
> - issue:elastic/integrations#17960 (`list_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
> - issue:elastic/integrations#17949 (`list_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
> - issue:elastic/integrations#17952 (`list_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
> - issue:elastic/integrations#17951 (`list_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
> - ... and 6 more items
>
>

---
[What is this?](https://ela.st/github-ai-tools) | [From workflow: Bug Hunter](https://github.com/elastic/integrations/actions/runs/24187538288)

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.
> - [x] expires on Apr 16, 2026, 11:33 AM UTC

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.