elastic / elastic/integrations

[bug-hunter] docs-edit-automation treats partial metadata updates as complete

Open Beginner friendly
#18,276 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Handlebars
Stars
333
Forks
647
Avg merge
2d 17h
Merged PRs (30d)
225

Description

## Impact
PRs that update package docs and **only one** metadata file (`manifest.yml` or `changelog.yml`) are marked as already complete by the docs helper workflow. That skips follow-up guidance even though one required metadata update is still missing, causing incomplete docs-update PRs and extra review churn.

## Reproduction Steps
1. Create and run this new minimal reproduction script:

```bash
cat > /tmp/gh-aw/agent/repro_docs_edit_automation.sh <<'EOF'
#!/usr/bin/env bash
set -euo pipefail

ALL_CHANGED_FILES=$'packages/sample/_dev/build/docs/README.md\npackages/sample/manifest.yml'
CHANGED_DOCS=$(echo "$ALL_CHANGED_FILES" | grep '^packages/.*/_dev/build/docs/README\.md$' || true)
PACKAGES_NEEDING_UPDATE=""
PACKAGES_ALREADY_DONE=""

while IFS= read -r doc_file; do
[ -z "$doc_file" ] && continue
PACKAGE=$(echo "$doc_file" | sed 's|^packages/\([^/]*\)/.*|\1|')
MANIFEST_CHANGED=$(echo "$ALL_CHANGED_FILES" | grep "^packages/$PACKAGE/manifest.yml$" || true)
CHANGELOG_CHANGED=$(echo "$ALL_CHANGED_FILES" | grep "^packages/$PACKAGE/changelog.yml$" || true)

if [ -z "$MANIFEST_CHANGED" ] && [ -z "$CHANGELOG_CHANGED" ]; then
PACKAGES_NEEDING_UPDATE="$PACKAGES_NEEDING_UPDATE $PACKAGE"
else
PACKAGES_ALREADY_DONE="$PACKAGES_ALREADY_DONE $PACKAGE"
fi
done <<< "$CHANGED_DOCS"

printf 'PACKAGES_NEEDING_UPDATE=%q\n' "$PACKAGES_NEEDING_UPDATE"
printf 'PACKAGES_ALREADY_DONE=%q\n' "$PACKAGES_ALREADY_DONE"
EOF
chmod +x /tmp/gh-aw/agent/repro_docs_edit_automation.sh
/tmp/gh-aw/agent/repro_docs_edit_automation.sh
```

## Expected vs Actual
**Expected:** when docs changed and only one of `manifest.yml` / `changelog.yml` changed, the package should still be in `PACKAGES_NEEDING_UPDATE`.

**Actual:** package is incorrectly marked done:

```text
PACKAGES_NEEDING_UPDATE=''
PACKAGES_ALREADY_DONE=\ sample
```

## Failing Test
```bash
#!/usr/bin/env bash
set -euo pipefail

ALL_CHANGED_FILES=$'packages/sample/_dev/build/docs/README.md\npackages/sample/manifest.yml'
CHANGED_DOCS=$(echo "$ALL_CHANGED_FILES" | grep '^packages/.*/_dev/build/docs/README\.md$' || true)
PACKAGES_NEEDING_UPDATE=""
PACKAGES_ALREADY_DONE=""

while IFS= read -r doc_file; do
[ -z "$doc_file" ] && continue
PACKAGE=$(echo "$doc_file" | sed 's|^packages/\([^/]*\)/.*|\1|')
MANIFEST_CHANGED=$(echo "$ALL_CHANGED_FILES" | grep "^packages/$PACKAGE/manifest.yml$" || true)
CHANGELOG_CHANGED=$(echo "$ALL_CHANGED_FILES" | grep "^packages/$PACKAGE/changelog.yml$" || true)

if [ -z "$MANIFEST_CHANGED" ] && [ -z "$CHANGELOG_CHANGED" ]; then
PACKAGES_NEEDING_UPDATE="$PACKAGES_NEEDING_UPDATE $PACKAGE"
else
PACKAGES_ALREADY_DONE="$PACKAGES_ALREADY_DONE $PACKAGE"
fi
done <<< "$CHANGED_DOCS"

# This assertion fails due to current workflow logic.
test -n "\$\{PACKAGES_NEEDING_UPDATE// }"
```

## Evidence
- Workflow logic classifies package as complete if **either** metadata file changed:
- `.github/workflows/docs-edit-automation.yml#L93-L101`
- Workflow summary text states completion means both were updated:
- `.github/workflows/docs-edit-automation.yml#L207` (`"updated changelogs and manifests"`)

The classification condition should require both files to be present before adding a package to `PACKAGES_ALREADY_DONE`.

> [!NOTE]
>
> 🔒 Integrity filtering filtered 31 items
>
> Integrity filtering activated and filtered the following items during workflow execution.
> This happens when a tool call accesses a resource that does not meet the required integrity or secrecy level of the workflow.
>
> - issue:elastic/integrations#unknown (`search_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
> - issue:elastic/integrations#18275 (`list_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
> - issue:elastic/integrations#18268 (`list_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
> - issue:elastic/integrations#18267 (`list_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
> - issue:elastic/integrations#18265 (`list_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
> - issue:elastic/integrations#18264 (`list_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
> - issue:elastic/integrations#18263 (`list_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
> - issue:elastic/integrations#18256 (`list_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
> - issue:elastic/integrations#18261 (`list_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
> - issue:elastic/integrations#18247 (`list_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
> - issue:elastic/integrations#18242 (`list_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
> - issue:elastic/integrations#18241 (`list_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
> - issue:elastic/integrations#18238 (`list_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
> - issue:elastic/integrations#18236 (`list_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
> - issue:elastic/integrations#18233 (`list_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
> - issue:elastic/integrations#18230 (`list_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
> - ... and 15 more items
>
>

---
[What is this?](https://ela.st/github-ai-tools) | [From workflow: Bug Hunter](https://github.com/elastic/integrations/actions/runs/24132704713)

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.
> - [x] expires on Apr 15, 2026, 11:33 AM UTC

Contributor guide

Open the contributing guide

Research direction

Start with .github/workflows/docs-edit-automation.yml around lines 93-101 and compare its metadata classification with the reproduction script in the issue. Run the provided script and failing assertion for the partial-update case, then verify cases with neither and both metadata files changed. Done means only packages with both manifest.yml and changelog.yml changes are marked complete.

Written by the indexing model from the issue text.

Assessment

Tech stack
bash, github-actions
Domain
ci-cd
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
88/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.