elastic / elastic/integrations
[bug-hunter] docs-edit-automation treats partial metadata updates as complete
- Dominant language
- Handlebars
- Stars
- 333
- Forks
- 647
- Avg merge
- 2d 17h
- Merged PRs (30d)
- 225
Description
## Impact
PRs that update package docs and **only one** metadata file (`manifest.yml` or `changelog.yml`) are marked as already complete by the docs helper workflow. That skips follow-up guidance even though one required metadata update is still missing, causing incomplete docs-update PRs and extra review churn.
## Reproduction Steps
1. Create and run this new minimal reproduction script:
```bash
cat > /tmp/gh-aw/agent/repro_docs_edit_automation.sh <<'EOF'
#!/usr/bin/env bash
set -euo pipefail
ALL_CHANGED_FILES=$'packages/sample/_dev/build/docs/README.md\npackages/sample/manifest.yml'
CHANGED_DOCS=$(echo "$ALL_CHANGED_FILES" | grep '^packages/.*/_dev/build/docs/README\.md$' || true)
PACKAGES_NEEDING_UPDATE=""
PACKAGES_ALREADY_DONE=""
while IFS= read -r doc_file; do
[ -z "$doc_file" ] && continue
PACKAGE=$(echo "$doc_file" | sed 's|^packages/\([^/]*\)/.*|\1|')
MANIFEST_CHANGED=$(echo "$ALL_CHANGED_FILES" | grep "^packages/$PACKAGE/manifest.yml$" || true)
CHANGELOG_CHANGED=$(echo "$ALL_CHANGED_FILES" | grep "^packages/$PACKAGE/changelog.yml$" || true)
if [ -z "$MANIFEST_CHANGED" ] && [ -z "$CHANGELOG_CHANGED" ]; then
PACKAGES_NEEDING_UPDATE="$PACKAGES_NEEDING_UPDATE $PACKAGE"
else
PACKAGES_ALREADY_DONE="$PACKAGES_ALREADY_DONE $PACKAGE"
fi
done <<< "$CHANGED_DOCS"
printf 'PACKAGES_NEEDING_UPDATE=%q\n' "$PACKAGES_NEEDING_UPDATE"
printf 'PACKAGES_ALREADY_DONE=%q\n' "$PACKAGES_ALREADY_DONE"
EOF
chmod +x /tmp/gh-aw/agent/repro_docs_edit_automation.sh
/tmp/gh-aw/agent/repro_docs_edit_automation.sh
```
## Expected vs Actual
**Expected:** when docs changed and only one of `manifest.yml` / `changelog.yml` changed, the package should still be in `PACKAGES_NEEDING_UPDATE`.
**Actual:** package is incorrectly marked done:
```text
PACKAGES_NEEDING_UPDATE=''
PACKAGES_ALREADY_DONE=\ sample
```
## Failing Test
```bash
#!/usr/bin/env bash
set -euo pipefail
ALL_CHANGED_FILES=$'packages/sample/_dev/build/docs/README.md\npackages/sample/manifest.yml'
CHANGED_DOCS=$(echo "$ALL_CHANGED_FILES" | grep '^packages/.*/_dev/build/docs/README\.md$' || true)
PACKAGES_NEEDING_UPDATE=""
PACKAGES_ALREADY_DONE=""
while IFS= read -r doc_file; do
[ -z "$doc_file" ] && continue
PACKAGE=$(echo "$doc_file" | sed 's|^packages/\([^/]*\)/.*|\1|')
MANIFEST_CHANGED=$(echo "$ALL_CHANGED_FILES" | grep "^packages/$PACKAGE/manifest.yml$" || true)
CHANGELOG_CHANGED=$(echo "$ALL_CHANGED_FILES" | grep "^packages/$PACKAGE/changelog.yml$" || true)
if [ -z "$MANIFEST_CHANGED" ] && [ -z "$CHANGELOG_CHANGED" ]; then
PACKAGES_NEEDING_UPDATE="$PACKAGES_NEEDING_UPDATE $PACKAGE"
else
PACKAGES_ALREADY_DONE="$PACKAGES_ALREADY_DONE $PACKAGE"
fi
done <<< "$CHANGED_DOCS"
# This assertion fails due to current workflow logic.
test -n "\$\{PACKAGES_NEEDING_UPDATE// }"
```
## Evidence
- Workflow logic classifies package as complete if **either** metadata file changed:
- `.github/workflows/docs-edit-automation.yml#L93-L101`
- Workflow summary text states completion means both were updated:
- `.github/workflows/docs-edit-automation.yml#L207` (`"updated changelogs and manifests"`)
The classification condition should require both files to be present before adding a package to `PACKAGES_ALREADY_DONE`.
> [!NOTE]
>
> 🔒 Integrity filtering filtered 31 items
>
> Integrity filtering activated and filtered the following items during workflow execution.
> This happens when a tool call accesses a resource that does not meet the required integrity or secrecy level of the workflow.
>
> - issue:elastic/integrations#unknown (`search_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
> - issue:elastic/integrations#18275 (`list_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
> - issue:elastic/integrations#18268 (`list_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
> - issue:elastic/integrations#18267 (`list_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
> - issue:elastic/integrations#18265 (`list_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
> - issue:elastic/integrations#18264 (`list_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
> - issue:elastic/integrations#18263 (`list_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
> - issue:elastic/integrations#18256 (`list_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
> - issue:elastic/integrations#18261 (`list_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
> - issue:elastic/integrations#18247 (`list_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
> - issue:elastic/integrations#18242 (`list_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
> - issue:elastic/integrations#18241 (`list_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
> - issue:elastic/integrations#18238 (`list_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
> - issue:elastic/integrations#18236 (`list_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
> - issue:elastic/integrations#18233 (`list_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
> - issue:elastic/integrations#18230 (`list_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
> - ... and 15 more items
>
>
---
[What is this?](https://ela.st/github-ai-tools) | [From workflow: Bug Hunter](https://github.com/elastic/integrations/actions/runs/24132704713)
Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.
> - [x] expires on Apr 15, 2026, 11:33 AM UTC
Contributor guide
Research direction
Start with .github/workflows/docs-edit-automation.yml around lines 93-101 and compare its metadata classification with the reproduction script in the issue. Run the provided script and failing assertion for the partial-update case, then verify cases with neither and both metadata files changed. Done means only packages with both manifest.yml and changelog.yml changes are marked complete.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- bash, github-actions
- Domain
- ci-cd
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 88/100