armbian / armbian/build

Artifacts with ENABLE_EXTENSIONS rebuild every run: predicted hooks-hash (-H) ≠ built hooks-hash

Open
#10,055 37 comments 0 reactions 0 assignees View on GitHub
Dominant language
Shell
Stars
5.4k
Forks
3.1k
Avg merge
1d 9h
Merged PRs (30d)
204

Description

## Summary

Any artifact whose board sets `ENABLE_EXTENSIONS` rebuilds on **every** matrix run and never caches. The artifact version's `-H` component (`hash_hooks_and_functions`) computed by the **info-gatherer** (`compile.sh gha-matrix`) differs from the one computed by the **actual artifact build** (`compile.sh artifact`) — at the *same framework commit*. So the build pushes one tag and `matrix_prep` forever checks for a different one → eternal cache miss.

## Evidence (uboot-bananapim7-vendor, BRANCH=vendor, ENABLE_EXTENSIONS=v4l2loopback-dkms)

At armbian/build main `9a6ea452`:

| phase | -H value | OCI state |
|---|---|---|
| `gha-matrix` predicts (matrix job name / OCI check) | `H40c4` | `ghcr.io/armbian/os/uboot-bananapim7-vendor:…-H40c4-…` → **404** |
| actual build produces + pushes (ORAS) | `H8240` | …`-H8240-…` → **200** |

The matrix predicts `H40c4`, checks OCI, gets 404, schedules a build. The build computes `H8240`, builds, pushes `…-H8240-…`. Next run: matrix predicts `H40c4` again → 404 → rebuild. Forever.

## Common factor

Only boards with `ENABLE_EXTENSIONS` are affected (e.g. `bananapim7,vendor` → `v4l2loopback-dkms,mesa-vpu`; `rock-5c,vendor` → `mesa-vpu`; the rk35xx vendor family generally). Boards with **no** extensions cache fine — their hooks set is empty so both phases agree.

## Root cause

`-H` = `sha256(hash_hooks + hash_uboot_functions)` ([artifact-uboot.sh#L85-L97](https://github.com/armbian/build/blob/main/lib/functions/artifacts/artifact-uboot.sh#L85-L97)). `hash_hooks` comes from `dump_extension_method_sources_functions`, which emits the **bodies of every `__` extension function currently *defined* in the bash process** (`compgen -A function`, [extensions.sh#L99](https://github.com/armbian/build/blob/main/lib/functions/general/extensions.sh#L99)).

The extension manager initializes **once per process** and refuses further loads ([extensions.sh#L140-L151](https://github.com/armbian/build/blob/main/lib/functions/general/extensions.sh#L140-L151)):
```bash
[[ ${initialize_extension_manager_counter} -lt 1 ]] && [[ "${ENABLE_EXTENSIONS}" != "" ]] && { … enable_extension … }
# "no more extensions are allowed to load after this."
```

So a clean per-board `artifact` build defines exactly that board's extension hooks → `H8240`, but the info-gatherer computes versions for many artifacts in a context whose enabled-extension set does **not** match each artifact's real build set → `H40c4`. Hence the predicted hooks-hash ≠ the built one for every extension-bearing artifact. (kernel/rootfs/bsp artifacts hash hooks the same way and are likely affected too.)

## Impact

Production is presumably silently rebuilding all extension-bearing artifacts every cycle (wasted runner time + churned OCI tags), in addition to the cache never warming for downstreams.

## Proposed fix direction

Compute each artifact's `hash_hooks` against **the exact extension set its build will use**, during info-gathering — i.e. derive the hooks-hash per artifact from its own `ENABLE_EXTENSIONS` (isolated extension state), rather than from whatever the info-gather process happened to load. Equivalently: the info-gather must reproduce, per artifact, the same defined `__` set the standalone build produces.

## Repro

`./compile.sh gha-matrix … TARGETS_FILTER_INCLUDE=BOARD:bananapim7` → note predicted `uboot-bananapim7-vendor=…-H-…`. Then run the produced `artifact` invocation → note pushed `…-H-…`. ` != `.

Contributor guide

Open the contributing guide

Research direction

Start with compile.sh gha-matrix and compile.sh artifact, then read lib/functions/artifacts/artifact-uboot.sh#L85-L97 and lib/functions/general/extensions.sh#L99-L151. Trace how matrix_prep and the standalone artifact build load ENABLE_EXTENSIONS and compute hash_hooks. Done means extension-bearing artifacts receive the same -H value in both phases and subsequent runs find the pushed OCI tag instead of rebuilding.

Written by the indexing model from the issue text.

Assessment

Tech stack
shell
Domain
build-system, ci-cd
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.