ethereum / ethereum/execution-specs
Pin EELS Commit in Hive Simulator Build Args for Reproducibility
- Dominant language
- Python
- Stars
- 1.2k
- Forks
- 505
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 116
Description
### Problem
The hive consume simulators accept a `branch` build argument (e.g., `--sim.buildarg branch=devnets/bal/3`) used in the simulator Dockerfile to check out EELS code.
This means every hive run builds from whatever `devnets/bal/3` points to at build time, **a moving target.** The reproduce commands shown on the hive results UI reflect this:
```
./hive --sim ethereum/eels/consume-rlp --client erigon --results-root results
--client-file=client-config.yaml --client.checktimelimit=300s --docker.buildoutput
--sim.parallelism=6
--sim.buildarg fixtures=https://github.com/ethereum/execution-spec-tests/releases/download/bal@v5.5.1/fixtures_bal.tar.gz
--sim.buildarg branch=devnets/bal/3
--sim.limit=.*fork_(Amsterdam|BPO2ToAmsterdamAtTime15k|Osaka).*
--sim.limit.exact=false --sim.loglevel=3
```
`--sim.buildarg branch=devnets/bal/3` has no commit pin. The hive UI tracks the **hive** commit (e.g., `77d3c12`) but **not** the EELS commit used to build the simulator. So the command shown is not fully reproducible — running it a day later may check out a different EELS commit and produce different results.
### Solution
Extend the `branch` build argument to optionally support a `branch@commit` format:
```
--sim.buildarg branch=devnets/bal/3@abc123def456
```
When `@commit` is provided, the Dockerfile checks out that exact commit. When omitted, behavior is unchanged (checks out branch HEAD). This keeps manual usage simple while making CI runs fully reproducible.
The reproduce command on the hive UI already passes through whatever `branch=` value hive was started with (via `HiveInfo.command`, `filtered_hive_options`, UI), so no changes needed in the EELS consume command generation code. The commit hash flows through automatically once CI includes it.
### Changes Required
#### 1. Hive, Simulator Dockerfiles ([`ethereum/hive`](https://github.com/ethereum/hive))
Parse the `branch` arg to extract an optional commit hash. All six Dockerfiles share the same git clone block and need the same update:
- `simulators/ethereum/eels/consume-engine/Dockerfile`
- `simulators/ethereum/eels/consume-enginex/Dockerfile`
- `simulators/ethereum/eels/consume-rlp/Dockerfile`
- `simulators/ethereum/eels/consume-sync/Dockerfile`
- `simulators/ethereum/eels/execute-blobs/Dockerfile`
- `simulators/ethereum/eels/Dockerfile.dev`
#### 2. Hive CI, Prepare job ([`ethpandaops/hive-tests`](https://github.com/ethpandaops/hive-tests))
Both BAL workflow files have a `prepare` job step (`Get latest bal fixtures and branch`) that dynamically resolves the highest-numbered `devnets/bal/N` branch but does **not** capture the commit hash:
```bash
# Current - no commit pin
BRANCH=$(gh api repos/ethereum/execution-specs/branches --paginate -q '.[].name' \
| grep -E '^devnets/bal/[0-9]+$' | sort -t/ -k3 -n | tail -1)
echo "eels_branch=$BRANCH" >> "$GITHUB_OUTPUT"
```
This needs to also resolve the HEAD commit of that branch and append it:
```bash
# Proposed - pin commit for reproducibility
BRANCH=$(gh api repos/ethereum/execution-specs/branches --paginate -q '.[].name' \
| grep -E '^devnets/bal/[0-9]+$' | sort -t/ -k3 -n | tail -1)
COMMIT=$(gh api "repos/ethereum/execution-specs/commits/${BRANCH}" -q '.sha' | head -c 12)
echo "eels_branch=${BRANCH}@${COMMIT}" >> "$GITHUB_OUTPUT"
```
Files to update:
- `.github/workflows/hive-devnet-3.yaml`
- `.github/workflows/hive-devnet-3-quick.yaml`
#### 3. EELS, Documentation
Update docs to describe the new `branch@commit` format:
- `docs/running_tests/hive/common_options.md`
- `docs/running_tests/hive/dev_mode.md`
No EELS code changes are needed.
### Benefits
- **Reproducible by default**: every CI hive run pins the exact EELS commit, and the reproduce command in the UI reflects it
- **Backward compatible**: `--sim.buildarg branch=devnets/bal/3` continues to work
- **Fast feedback preserved**: CI still tracks the latest devnet branch automatically, it just records which commit it built from
Contributor guide
Assessment
This issue has not been assessed yet.