elastic / elastic/ai-github-actions
[autonomy-atomicity] Lockfile-coupled label-sanitize test is a parallel-change hotspot
- Dominant language
- Python
- Stars
- 11
- Forks
- 16
- Avg merge
- 22h 9m
- Merged PRs (30d)
- 31
Description
## Autonomy / Atomicity Findings
### 1. Lockfile-coupled add-labels regression test creates cross-workflow contention
**Category:** Over-broad test
**File(s):** `tests/test_safe_output_add_labels.py` (lines 29-33, 195-204, 46-52)
**Problem:**
`tests/test_safe_output_add_labels.py` validates three generated lockfiles (`gh-aw-dependency-review.lock.yml`, `gh-aw-issue-triage.lock.yml`, `gh-aw-pr-labeler.lock.yml`) by asserting the presence/absence of exact sanitize-script text patterns. This couples one test to compiler output shape across multiple workflows instead of validating behavior only.
Concrete evidence from `tests/test_safe_output_add_labels.py`:
- `LOCK_FILES = [...]` includes three `.lock.yml` artifacts (lines 29-33)
- `assert "typeof v === 'object'" in text` (line 198)
- `assert not re.search(r"\.map\(\(v\) => String\(v\)\.trim\(\)\)", text, ...)` (lines 201-204)
When two concurrent changes land (for example, a compiler-emitted formatting change in lockfiles and an unrelated workflow update), this test can fail even if label sanitization behavior is still correct. It becomes a merge/rebase hotspot and creates noisy breakage across unrelated workstreams.
**Suggested fix:**
Refocus this check on behavior and authored sources:
1. Keep behavior tests that execute sanitize scripts (already done for source fragments in this test file).
2. Replace exact lockfile text assertions with structural extraction from source workflow markdown/frontmatter only, or remove lockfile text coupling entirely.
3. If compiled artifacts must be checked, assert a semantic invariant from parsed YAML/script blocks rather than brittle string patterns.
## Suggested Actions
- [ ] Remove direct string-pattern assertions over generated `.lock.yml` files from `test_compiled_locks_resolve_object_labels`.
- [ ] Validate object-label sanitization through executable script behavior tests for source workflow definitions.
- [ ] Add a narrow compile-integrity assertion (semantic, not textual) only if needed to guard regression risk.
---
[What is this?](https://ela.st/github-ai-tools) | [From workflow: Trigger Autonomy Atomicity Analyzer](https://github.com/elastic/ai-github-actions/actions/runs/34375649622)
Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.
Contributor guide
Research direction
Start with tests/test_safe_output_add_labels.py, especially test_compiled_locks_resolve_object_labels and the LOCK_FILES/assertions around lines 29-33 and 195-204. Run the focused test first, then review the existing executable sanitize-script behavior checks. Done means generated .lock.yml files are no longer checked with brittle string patterns, while label-sanitization behavior remains covered and any compile check is semantic.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- testing
- Issue type
- Refactor
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 74/100