ChainSafe / ChainSafe/lodestar
compliance fork-choice tests: gloas SSZ types diverge from consensus-specs master
- Dominant language
- TypeScript
- Stars
- 1.4k
- Forks
- 483
- Avg merge
- 1d 18h
- Merged PRs (30d)
- 156
Description
## Context
Discovered while wiring up the consensus-specs fork-choice compliance test runner in #9290. With the runner in place, **1280 / 2944** compliance tests fail at SSZ load time before any logic runs. All affected cases are gloas suites except `block_cover_test`, which doesn't include the divergent fixture files.
## Root cause
Lodestar's `packages/beacon-node/test/spec-tests-version.json` pins consensus-specs to `v1.7.0-alpha.5` — the latest available release. The compliance test artifact, however, is generated by the consensus-specs **Compliance Tests** workflow which builds from `master` (e.g. commit `ccc96eb`). Master is ahead of any released alpha, so the gloas SSZ container shapes have drifted vs Lodestar's `@lodestar/types` definitions.
`v1.7.0-alpha.5` was released 2026-04-19; everything between then and `master` HEAD is unreleased.
## Concrete divergences
Bisected via a small probe script (below) that snappy-decompresses each `.ssz_snappy` file in a failing case and tries to deserialize it with the type the runner registers.
Test case used: `spec-tests-compliance/small/tests/minimal/gloas/fork_choice_compliance/attester_slashing_test/pyspec_tests/attester_slashing_test_0_10950258_0`
| File | Lodestar type | Status | Error |
|---|---|---|---|
| `anchor_block.ssz_snappy` | `ssz.gloas.BeaconBlock` | FAIL | `First offset must equal to fixedEnd 336 != 396` (+60 bytes) |
| `block_.ssz_snappy` | `ssz.gloas.SignedBeaconBlock` | FAIL | same `336 != 396` |
| `attestation_.ssz_snappy` | `ssz.gloas.Attestation` | FAIL | `229 != 236` (+7 bytes) |
| `execution_payload_envelope_.ssz_snappy` | `ssz.gloas.SignedExecutionPayloadEnvelope` | FAIL | `80 != 48` (-32 bytes) |
| `anchor_state.ssz_snappy` | `ssz.gloas.BeaconState` | FAIL | `Offset is outside the bounds of the DataView` (cascading from one of the above) |
| `attester_slashing_.ssz_snappy` | `ssz.gloas.AttesterSlashing` | OK | – |
| `payload_attestation_.ssz_snappy` | _none registered_ | SKIP | new gloas type not yet in `@lodestar/types` |
The gaps are different across types, so this isn't a single field rename — multiple gloas containers have changed on master.
## Probe script
Drop into `packages/beacon-node/probe.mjs` and run with `node probe.mjs` after extracting compliance test data. Useful for re-checking once a new spec release lands and `spec-tests-version.json` is bumped.
```js
import fs from "node:fs";
import path from "node:path";
import {uncompress} from "snappyjs";
import {ssz} from "@lodestar/types";
const CASE = "spec-tests-compliance/small/tests/minimal/gloas/fork_choice_compliance/attester_slashing_test/pyspec_tests/attester_slashing_test_0_10950258_0";
const types = {
anchor_state: ssz.gloas.BeaconState,
anchor_block: ssz.gloas.BeaconBlock,
"^(block)_([0-9a-zA-Z]+)$": ssz.gloas.SignedBeaconBlock,
"^(execution_payload_envelope)_([0-9a-zA-Z]+)$": ssz.gloas.SignedExecutionPayloadEnvelope,
"^(attestation)_([0-9a-zA-Z])+$": ssz.gloas.Attestation,
"^(attester_slashing)_([0-9a-zA-Z])+$": ssz.gloas.AttesterSlashing,
};
const matches = (name, k) => k.startsWith("^") ? name.match(k) : name === k;
for (const f of fs.readdirSync(CASE).filter((f) => f.endsWith(".ssz_snappy")).sort()) {
const inputName = f.replace(".ssz_snappy", "");
const k = Object.keys(types).find((k) => matches(inputName, k));
if (!k) { console.log(`SKIP ${f} (no registered type)`); continue; }
const bytes = uncompress(fs.readFileSync(path.join(CASE, f)));
try {
types[k].deserialize(bytes);
console.log(`OK ${f} type=${types[k].typeName}`);
} catch (e) {
console.log(`FAIL ${f} type=${types[k].typeName} -- ${e.message}`);
}
}
```
## Suggested resolution
1. Wait for the next `consensus-specs` alpha release (`v1.7.0-alpha.6` or later) that captures these gloas changes.
2. Bump `packages/beacon-node/test/spec-tests-version.json` to that release.
3. Update `@lodestar/types` for any container shape changes the new alpha includes (`BeaconBlock`/`BeaconBlockBody`, `Attestation`, `ExecutionPayloadEnvelope`, plus the new `PayloadAttestation` type).
4. Re-run `./scripts/compliance-fc-report.sh` and confirm the `1280x` SSZ-deserialize failure class is gone.
Attempting (3) ahead of a release would be premature — the spec is still moving on master and our types could go stale within days.
## Impact
Until resolved, `~13%` (gloas) of the compliance pass rate is gated on this. Fulu side is unaffected and currently passes ~21% — see #9290 for the full per-suite breakdown.
Contributor guide
Research direction
Start with packages/beacon-node/test/spec-tests-version.json and wait for a consensus-specs alpha release containing the described gloas changes. Then inspect @lodestar/types for BeaconBlock/BeaconBlockBody, Attestation, ExecutionPayloadEnvelope, and PayloadAttestation, and run ./scripts/compliance-fc-report.sh. Done means the 1280x SSZ-deserialization failure class is gone while the unaffected suites still pass.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- node.js, typescript
- Domain
- blockchain, testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100