ethereum-optimism / ethereum-optimism/optimism

flake: TestSupernodeLightSequencerInteropInvalidMessageReplacement

Open
#22,234 4 comments 0 reactions 1 assignee Claimed by @einar-oplabs View on GitHub
Dominant language
Go
Stars
6.5k
Forks
4k
Avg merge
2d 18h
Merged PRs (30d)
134

Description

## Description

`TestSupernodeLightSequencerInteropInvalidMessageReplacement` (`op-acceptance-tests/tests/supernode/interop/reorg/invalid_message_reorg_test.go`) failed once during memory-all acceptance measurement runs on 2026-08-04 and passed in the seven sibling runs executed the same day across two concurrency shapes.

## Occurrence

- Job: `memory-all-opn-op-reth-prague` (4 nodes × `ACCEPTANCE_TEST_JOBS=8`), CircleCI job 5423722 on `einar/ci-timing-splits` (#22202 measurement rerun)
- Test runtime: 71s, result `failure`; the run completed 85 tests with this single failure
- Same test green in: 3× other 4×8 runs and 4× 4×16 runs that day
- No existing `MarkFlaky` marker and no prior issue found for this test name

## Status

- #22235 (mark the test flaky) was **closed unmerged**: at ~2 failures in 7 days — a rate many tests currently show — the cause looks environmental rather than specific to this test, and `MarkFlaky` would ignore the test until someone fixes it, which is the wrong trade here. `MarkFlaky` should be a last resort, not the default response to an occasional failure.

## Next steps

- **Root cause found** — see [the analysis comment](https://github.com/ethereum-optimism/optimism/issues/22234#issuecomment-5215613729): a real race in the test's final assertions (flake-catalogue shape F4), not environmental. The `Reached(eth.Safe, N)` gate checks the safe label number while `AssertTxInBlock(N)` reads by number against the canonical chain, which transiently flips back to the light sequencer's divergent branch when an in-flight build job seals on a stale parent after a follow-source reorg (#21119 oscillation)
- Fix the test: #22321 adds retrying tx-in-block/tx-not-in-block checks on the exact postcondition instead of gating on the safe label
- The third occurrence (build 5418160) was a genuine post-replacement cross-safe wedge, not a budget problem — split into #22325

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.