elastic / elastic/ai-github-actions

[framework-best-practices] Keep gh-aw compilation and smoke tests aligned with current compiler failures

Open Beginner friendly
#1,670 1 comment 0 reactions 0 assignees View on GitHub
framework-best-practices
Dominant language
Python
Stars
11
Forks
16
Avg merge
22h 9m
Merged PRs (30d)
31

Description

## Framework / Library Best Practices Findings

### 1. Make ignores gh-aw compiler failures
**Library/tool:** GNU Make and the gh-aw compiler
**Location:** `Makefile:193-195`
**Current code:** The `compile` target invokes `.bin/gh-aw compile` as `-@.bin/gh-aw compile`; GNU Make treats the leading `-` as an instruction to ignore a non-zero exit status.
**What is wrong:** A compiler error can be printed while `make compile` continues into post-processing and exits successfully, allowing stale or incomplete generated lock files to appear valid to CI or maintainers.
**Why it matters:** Workflow source changes can fail to compile without stopping the required generation/validation pipeline, so the repository can publish broken compiled workflows or conceal the actual failure.
**Simplification:** Remove the leading `-` so the existing compiler exit status is enforced by Make. GNU Make documents that prefixing a recipe line with `-` ignores errors: (www.gnu.org/redacted)

### 2. The smoke test installs an obsolete compiler version
**Library/tool:** GitHub Agent Workflows (`gh-aw`)
**Locations:** `Makefile:4` pins the repository compiler to `v0.82.10`; `.github/workflows/smoke-test-install.yml:15-28` pins the daily smoke test to `v0.48.1`.
**What is wrong:** The smoke test validates installation and compilation using a compiler that is 34 releases behind the version used by the repository’s compile target. It therefore does not exercise the current workflow syntax, compiler behavior, or generated output.
**Why it matters:** Regressions introduced by the current compiler can pass the only scheduled install/compile check because that check is testing a different toolchain.
**Simplification:** Define the smoke-test version from the same single source used by `make compile` (or update it whenever `GH_AW_VERSION` changes), then compile with that version.

## Suggested Actions
- [ ] Remove the error-suppression prefix from the `gh-aw compile` recipe and ensure compiler failures fail the target.
- [ ] Align `.github/workflows/smoke-test-install.yml` with the repository’s current `GH_AW_VERSION` instead of maintaining the stale `v0.48.1` pin.

> [!WARNING]
>
> Firewall blocked 1 domain
>
> The following domain was blocked by the firewall during workflow execution:
>
> - `awmgmcpg`
>> To allow these domains, add them to the `network.allowed` list in your workflow frontmatter:
>
> ```yaml
> network:
> allowed:
> - defaults
> - "awmgmcpg"
> ```
>
> See [Network Configuration](https://github.github.com/gh-aw/reference/network/) for more information.
>
>

---
[What is this?](https://ela.st/github-ai-tools) | [From workflow: Trigger Framework Best Practices](https://github.com/elastic/ai-github-actions/actions/runs/30097343908)

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.

Contributor guide

Open the contributing guide

Research direction

Start in Makefile:4 and the compile recipe at lines 193-195, then compare them with .github/workflows/smoke-test-install.yml:15-28. Verify that compiler failures stop the Make target and that the smoke test uses the repository's current GH_AW_VERSION. Run the affected compile and smoke-test workflows to confirm both checks exercise the same compiler version.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions
Domain
build-system, ci-cd
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.