ChainSafe / ChainSafe/lodestar

compliance fork-choice tests: gloas SSZ types diverge from consensus-specs master

Open
#9,295 0 comments 0 reactions 0 assignees View on GitHub
spec-gloas
Dominant language
TypeScript
Stars
1.4k
Forks
483
Avg merge
1d 18h
Merged PRs (30d)
156

Description

## Context

Discovered while wiring up the consensus-specs fork-choice compliance test runner in #9290. With the runner in place, **1280 / 2944** compliance tests fail at SSZ load time before any logic runs. All affected cases are gloas suites except `block_cover_test`, which doesn't include the divergent fixture files.

## Root cause

Lodestar's `packages/beacon-node/test/spec-tests-version.json` pins consensus-specs to `v1.7.0-alpha.5` — the latest available release. The compliance test artifact, however, is generated by the consensus-specs **Compliance Tests** workflow which builds from `master` (e.g. commit `ccc96eb`). Master is ahead of any released alpha, so the gloas SSZ container shapes have drifted vs Lodestar's `@lodestar/types` definitions.

`v1.7.0-alpha.5` was released 2026-04-19; everything between then and `master` HEAD is unreleased.

## Concrete divergences

Bisected via a small probe script (below) that snappy-decompresses each `.ssz_snappy` file in a failing case and tries to deserialize it with the type the runner registers.

Test case used: `spec-tests-compliance/small/tests/minimal/gloas/fork_choice_compliance/attester_slashing_test/pyspec_tests/attester_slashing_test_0_10950258_0`

| File | Lodestar type | Status | Error |
|---|---|---|---|
| `anchor_block.ssz_snappy` | `ssz.gloas.BeaconBlock` | FAIL | `First offset must equal to fixedEnd 336 != 396` (+60 bytes) |
| `block_.ssz_snappy` | `ssz.gloas.SignedBeaconBlock` | FAIL | same `336 != 396` |
| `attestation_.ssz_snappy` | `ssz.gloas.Attestation` | FAIL | `229 != 236` (+7 bytes) |
| `execution_payload_envelope_.ssz_snappy` | `ssz.gloas.SignedExecutionPayloadEnvelope` | FAIL | `80 != 48` (-32 bytes) |
| `anchor_state.ssz_snappy` | `ssz.gloas.BeaconState` | FAIL | `Offset is outside the bounds of the DataView` (cascading from one of the above) |
| `attester_slashing_.ssz_snappy` | `ssz.gloas.AttesterSlashing` | OK | – |
| `payload_attestation_.ssz_snappy` | _none registered_ | SKIP | new gloas type not yet in `@lodestar/types` |

The gaps are different across types, so this isn't a single field rename — multiple gloas containers have changed on master.

## Probe script

Drop into `packages/beacon-node/probe.mjs` and run with `node probe.mjs` after extracting compliance test data. Useful for re-checking once a new spec release lands and `spec-tests-version.json` is bumped.

```js
import fs from "node:fs";
import path from "node:path";
import {uncompress} from "snappyjs";
import {ssz} from "@lodestar/types";

const CASE = "spec-tests-compliance/small/tests/minimal/gloas/fork_choice_compliance/attester_slashing_test/pyspec_tests/attester_slashing_test_0_10950258_0";

const types = {
anchor_state: ssz.gloas.BeaconState,
anchor_block: ssz.gloas.BeaconBlock,
"^(block)_([0-9a-zA-Z]+)$": ssz.gloas.SignedBeaconBlock,
"^(execution_payload_envelope)_([0-9a-zA-Z]+)$": ssz.gloas.SignedExecutionPayloadEnvelope,
"^(attestation)_([0-9a-zA-Z])+$": ssz.gloas.Attestation,
"^(attester_slashing)_([0-9a-zA-Z])+$": ssz.gloas.AttesterSlashing,
};
const matches = (name, k) => k.startsWith("^") ? name.match(k) : name === k;

for (const f of fs.readdirSync(CASE).filter((f) => f.endsWith(".ssz_snappy")).sort()) {
const inputName = f.replace(".ssz_snappy", "");
const k = Object.keys(types).find((k) => matches(inputName, k));
if (!k) { console.log(`SKIP ${f} (no registered type)`); continue; }
const bytes = uncompress(fs.readFileSync(path.join(CASE, f)));
try {
types[k].deserialize(bytes);
console.log(`OK ${f} type=${types[k].typeName}`);
} catch (e) {
console.log(`FAIL ${f} type=${types[k].typeName} -- ${e.message}`);
}
}
```

## Suggested resolution

1. Wait for the next `consensus-specs` alpha release (`v1.7.0-alpha.6` or later) that captures these gloas changes.
2. Bump `packages/beacon-node/test/spec-tests-version.json` to that release.
3. Update `@lodestar/types` for any container shape changes the new alpha includes (`BeaconBlock`/`BeaconBlockBody`, `Attestation`, `ExecutionPayloadEnvelope`, plus the new `PayloadAttestation` type).
4. Re-run `./scripts/compliance-fc-report.sh` and confirm the `1280x` SSZ-deserialize failure class is gone.

Attempting (3) ahead of a release would be premature — the spec is still moving on master and our types could go stale within days.

## Impact

Until resolved, `~13%` (gloas) of the compliance pass rate is gated on this. Fulu side is unaffected and currently passes ~21% — see #9290 for the full per-suite breakdown.

Contributor guide

Open the contributing guide

Research direction

Start with packages/beacon-node/test/spec-tests-version.json and wait for a consensus-specs alpha release containing the described gloas changes. Then inspect @lodestar/types for BeaconBlock/BeaconBlockBody, Attestation, ExecutionPayloadEnvelope, and PayloadAttestation, and run ./scripts/compliance-fc-report.sh. Done means the 1280x SSZ-deserialization failure class is gone while the unaffected suites still pass.

Written by the indexing model from the issue text.

Assessment

Tech stack
node.js, typescript
Domain
blockchain, testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.