elastic / elastic/ai-github-actions
[framework-best-practices] Keep gh-aw compilation and smoke tests aligned with current compiler failures
- Dominant language
- Python
- Stars
- 11
- Forks
- 16
- Avg merge
- 22h 9m
- Merged PRs (30d)
- 31
Description
## Framework / Library Best Practices Findings
### 1. Make ignores gh-aw compiler failures
**Library/tool:** GNU Make and the gh-aw compiler
**Location:** `Makefile:193-195`
**Current code:** The `compile` target invokes `.bin/gh-aw compile` as `-@.bin/gh-aw compile`; GNU Make treats the leading `-` as an instruction to ignore a non-zero exit status.
**What is wrong:** A compiler error can be printed while `make compile` continues into post-processing and exits successfully, allowing stale or incomplete generated lock files to appear valid to CI or maintainers.
**Why it matters:** Workflow source changes can fail to compile without stopping the required generation/validation pipeline, so the repository can publish broken compiled workflows or conceal the actual failure.
**Simplification:** Remove the leading `-` so the existing compiler exit status is enforced by Make. GNU Make documents that prefixing a recipe line with `-` ignores errors: (www.gnu.org/redacted)
### 2. The smoke test installs an obsolete compiler version
**Library/tool:** GitHub Agent Workflows (`gh-aw`)
**Locations:** `Makefile:4` pins the repository compiler to `v0.82.10`; `.github/workflows/smoke-test-install.yml:15-28` pins the daily smoke test to `v0.48.1`.
**What is wrong:** The smoke test validates installation and compilation using a compiler that is 34 releases behind the version used by the repository’s compile target. It therefore does not exercise the current workflow syntax, compiler behavior, or generated output.
**Why it matters:** Regressions introduced by the current compiler can pass the only scheduled install/compile check because that check is testing a different toolchain.
**Simplification:** Define the smoke-test version from the same single source used by `make compile` (or update it whenever `GH_AW_VERSION` changes), then compile with that version.
## Suggested Actions
- [ ] Remove the error-suppression prefix from the `gh-aw compile` recipe and ensure compiler failures fail the target.
- [ ] Align `.github/workflows/smoke-test-install.yml` with the repository’s current `GH_AW_VERSION` instead of maintaining the stale `v0.48.1` pin.
> [!WARNING]
>
> Firewall blocked 1 domain
>
> The following domain was blocked by the firewall during workflow execution:
>
> - `awmgmcpg`
>> To allow these domains, add them to the `network.allowed` list in your workflow frontmatter:
>
> ```yaml
> network:
> allowed:
> - defaults
> - "awmgmcpg"
> ```
>
> See [Network Configuration](https://github.github.com/gh-aw/reference/network/) for more information.
>
>
---
[What is this?](https://ela.st/github-ai-tools) | [From workflow: Trigger Framework Best Practices](https://github.com/elastic/ai-github-actions/actions/runs/30097343908)
Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.
Contributor guide
Research direction
Start in Makefile:4 and the compile recipe at lines 193-195, then compare them with .github/workflows/smoke-test-install.yml:15-28. Verify that compiler failures stop the Make target and that the smoke test uses the repository's current GH_AW_VERSION. Run the affected compile and smoke-test workflows to confirm both checks exercise the same compiler version.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- github-actions
- Domain
- build-system, ci-cd
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 72/100