elastic / elastic/ai-github-actions

[autonomy-atomicity] Brittle post-compile patching depends on exact lockfile formatting

Open
#1,859 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
11
Forks
16
Avg merge
22h 9m
Merged PRs (30d)
31

Description

## Autonomy / Atomicity Findings

### 1. Post-compile wiring relies on exact text match in generated lockfiles
**Category:** Ordering dependency
**File(s):**
- `scripts/wire-report-failure-input.sh` (lines 23-35, 37-45)
- `Makefile` (lines 193-207)

**Problem:**
The `compile` target always runs `./scripts/wire-report-failure-input.sh` after generating lockfiles. That script only rewrites files when it finds one exact line:

```bash
OLD=' GH_AW_FAILURE_REPORT_AS_ISSUE: "true"'
...
if ! grep -qF "$OLD" "$lock_file"; then
continue
fi
...
if [ "$line" = "$OLD" ]; then
printf '%s\n' "$NEW"
fi
```

This creates a hidden ordering/format dependency on the compiler's current YAML rendering (indentation, quoting, or line formatting). If a compiler upgrade or parallel change alters formatting but not semantics, the patch silently skips affected lockfiles.

**Why this blocks parallel development:**
Two independent changes can land correctly in isolation but break together:
1. one PR upgrades or changes compiler output formatting;
2. another PR relies on `report-failure-as-issue` wiring behavior.

After merge, the post-process step may no-op without error, leaving hardcoded `"true"` values in some lockfiles and causing inconsistent failure-report behavior. This is a subtle integration hazard that is hard to detect in review.

**Suggested fix:**
- Replace line-exact string patching with structured YAML mutation (key-path based), or emit the desired expression directly from workflow sources/fragments so no post-hoc textual rewrite is needed.
- Add a hard validation step after patching that fails when any lockfile containing `report-failure-as-issue:` still contains the hardcoded `GH_AW_FAILURE_REPORT_AS_ISSUE: "true"` value.

## Suggested Actions
- [ ] Refactor `scripts/wire-report-failure-input.sh` to use structure-aware rewriting instead of exact line matching.
- [ ] Add a compile-time validation check that fails on unwired `GH_AW_FAILURE_REPORT_AS_ISSUE` entries.
- [ ] Consider moving this wiring into source workflow/fragment generation to eliminate fragile postprocessing.

---
[What is this?](https://ela.st/github-ai-tools) | [From workflow: Trigger Autonomy Atomicity Analyzer](https://github.com/elastic/ai-github-actions/actions/runs/32158696138)

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.

Contributor guide

Open the contributing guide

Research direction

Start with scripts/wire-report-failure-input.sh and the compile target in Makefile, then run the compile flow to observe how generated lockfiles are patched. Trace the report-failure-as-issue entries and compare the current exact-line behavior with the proposed validation requirements. Done means formatting changes no longer silently skip wiring and compile-time validation detects any remaining hardcoded GH_AW_FAILURE_REPORT_AS_ISSUE values.

Written by the indexing model from the issue text.

Assessment

Tech stack
bash
Domain
build-system
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.