midnightntwrk / midnightntwrk/midnight-node

devnet: 2.1.0 ships the new, rebuilt genesis — HF 1.0.300 -> 2.1.0 nodes from an empty disk cannot join the live chain

Open
#2,153 3 comments 0 reactions 1 assignee View on GitHub

@rsporny is already working on this.

Since Sep 15, 2026.

bot:ai-assisted component:midnight-node
Dominant language
Rust
Stars
68
Forks
45
Avg merge
2d 1h
Merged PRs (30d)
64

Description

Context & versions

This is operational, not a code defect. The devnet genesis was legitimately rebuilt on release/node-2.1.0 because devnet was reset onto 2.1.0. However, devnet was repurposed to test the hard fork, and was started from 1.0.300 and upgraded in place via runtime upgrade. The live chain therefore still runs the 1.0.300-era genesis, while the 2.1.0 release artifacts carry the genesis built for the reset that was cancelled.

  • Node: 2.1.0-4a9eda5d, stl-k8s-devnet-validator-07 (k8s, ParityDb)
  • Network: devnet — 7 validators + RPC node; live chain started from 1.0.300-8f379791
  • Chain spec: Midnight devnet / midnight_devnet
  • Frequency: deterministic — every start from an empty disk
Steps to reproduce
  1. Start a 2.1.0 node against devnet with an empty data volume (CFG_PRESET=devnet).
  2. Watch peering.
Expected behavior

The node computes the same genesis as the running devnet chain, peers with the bootnodes, and syncs.

Actual behavior

The node initialises a genesis that no peer recognises. All five bootnodes are rejected as "different chain" and banned, leaving it at block #0 with zero peers indefinitely.

10:53:48  🔨 Initializing Genesis block/state (state: 0xcf3e…764c, header-hash: 0x6219…033c)
10:53:48  ERROR sync: Bootnode with peer id 12D3KooWEoaZD8tKtqdbAT72fvYKcUCp4HGHBetZK4vF9kVdz2fw
                 is on a different chain (our genesis: 0x6219…033c theirs: 0xe50e…60e5)
10:53:53  WARN  peerset: Report 12D3KooWEoaZD8tKtqdbAT72fvYKcUCp4HGHBetZK4vF9kVdz2fw:
                 -2147483648 to -2147483648. Reason: Genesis mismatch. Banned, disconnecting.
          … all five bootnodes, cycle repeats …
10:55:23  💤 Idle (0 peers), best: #0 (0x6219…033c), finalized #0 (0x6219…033c)
genesis computed by the 2.1.0 node 0x6219…033c (state 0xcf3e…764c)
genesis of the running devnet chain 0xe50e…60e5
Which artifacts differ

Comparing the devnet genesis inputs on release/node-2.1.0 against the 1.0.300 tree (release/node-1.0.300 @ cba37cf7adfa in the shadow fork), 7 of the 11 inputs that feed load_spec differ:

Input State
res/genesis/genesis_state_devnet.mn differs
res/genesis/genesis_block_devnet.mn differs
res/devnet/cnight-config.json differs
res/devnet/ics-config.json differs
res/devnet/reserve-config.json differs
res/devnet/system-parameters-config.json differs
res/devnet/permissioned-candidates-config.json differs
res/devnet/c2m-bridge-config.json new in 2.1.0 — no 1.0.300 counterpart, and load_spec requires it (c2m_bridge_config not configured)
res/devnet/pc-chain-config.json same
res/devnet/federated-authority-config.json same
res/devnet/registered-candidates-addresses.json same

The generated res/devnet/chain-spec-raw.json shows the scale of the difference — the genesis state is regenerated wholesale, not edited:

1.0.300 2.1.0
genesis.raw.top entries 256 2,910
keys added 2,768
keys removed 114
values changed 15

chainspec_name and chainspec_id are unchanged, so the spec still identifies as the same network while describing a different chain. res/devnet/chain-spec-hash.json (new in 2.1.0) pins 73e72fd2…, matching the 2.1.0 raw spec — so the build is self-consistently pinned to the spec that does not match the live chain.

Impact

Blocks every from-empty-disk bring-up on devnet with 2.1.0: new validator onboarding, disaster recovery, and any from-genesis sync test. Nodes that took the runtime upgrade in place are unaffected — they keep the original genesis on disk.

This blocks the remaining post-upgrade test passes (wallet, DUST, governance, transaction, cNIGHT bridge, security patch), all of which need a node that can sync.

Workaround

None for a fresh node. Existing nodes keep running because their on-disk genesis predates the 2.1.0 artifacts — so do not wipe a working devnet node's volume until this is resolved.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.