ethereum-optimism / ethereum-optimism/optimism
op-acceptance-tests: CI flakiness while initialization
- Dominant language
- Go
- Stars
- 6.5k
- Forks
- 4k
- Avg merge
- 2d 15h
- Merged PRs (30d)
- 145
Description
https://app.circleci.com/jobs/github/ethereum-optimism/optimism/3776821 shows flaky failures.
All this issues were observed before, but never been tracked. At first glance all issues stems while initialization the tests, before running the actual test logic.
Two buckets of issues:
1. 404 error
```
=== RUN TestFaucetFund
l2.go:389: �[32mINFO �[0m[08-15|10:51:47.932] Loaded devnet keys: 12 L1 system keys, 30 L2 user keys
l2.go:389: �[32mINFO �[0m[08-15|10:51:47.935] Loaded devnet keys: 12 L1 system keys, 30 L2 user keys
assertions.go:369: �[31mERROR�[0m[08-15|10:51:51.176]
assertions.go:369: Error Trace: /home/circleci/project/op-devstack/dsl/eoa.go:117
assertions.go:369: /home/circleci/project/op-devstack/dsl/eoa.go:123
assertions.go:369: /home/circleci/project/op-acceptance-tests/tests/base/faucet_test.go:19
assertions.go:369: Error: Received unexpected error:
assertions.go:369: operation failed permanently after 3 attempts: 404 Not Found: 404 page not found
```
```
=== RUN TestFaucetFund
l2.go:389: �[32mINFO �[0m[08-15|10:51:47.932] Loaded devnet keys: 12 L1 system keys, 30 L2 user keys
l2.go:389: �[32mINFO �[0m[08-15|10:51:47.935] Loaded devnet keys: 12 L1 system keys, 30 L2 user keys
assertions.go:369: �[31mERROR�[0m[08-15|10:51:51.176]
assertions.go:369: Error Trace: /home/circleci/project/op-devstack/dsl/eoa.go:117
assertions.go:369: /home/circleci/project/op-devstack/dsl/eoa.go:123
assertions.go:369: /home/circleci/project/op-acceptance-tests/tests/base/faucet_test.go:19
assertions.go:369: Error: Received unexpected error:
assertions.go:369: operation failed permanently after 3 attempts: 404 Not Found: 404 page not found
assertions.go:369: Test: TestFaucetFund
assertions.go:369: Messages: must lookup balance
assertions.go:369:
--- FAIL: TestFaucetFund (3.25s)
```
2. Waiting for online error
```
=== RUN TestL1ToL2Deposit
l2.go:389: �[32mINFO �[0m[08-15|10:52:39.909] Loaded devnet keys: 12 L1 system keys, 30 L2 user keys
l2.go:389: �[32mINFO �[0m[08-15|10:52:39.911] Loaded devnet keys: 12 L1 system keys, 30 L2 user keys
el.go:61: �[32mINFO �[0m[08-15|10:52:40.415] Waiting for online
el.go:61: �[32mINFO �[0m[08-15|10:52:40.915] Waiting for online
el.go:61: �[32mINFO �[0m[08-15|10:52:41.415] Waiting for online
...
el.go:61: �[32mINFO �[0m[08-15|10:52:48.915] Waiting for online
el.go:61: �[32mINFO �[0m[08-15|10:52:49.415] Waiting for online
assertions.go:369: �[31mERROR�[0m[08-15|10:52:49.915]
assertions.go:369: Error Trace: /home/circleci/project/op-service/testreq/require.go:64
assertions.go:369: /home/circleci/project/op-devstack/dsl/el.go:60
assertions.go:369: /home/circleci/project/op-devstack/dsl/l1_network.go:67
assertions.go:369: /home/circleci/project/op-acceptance-tests/tests/base/deposit/deposit_test.go:27
assertions.go:369: Error: Condition never satisfied
assertions.go:369: Test: TestL1ToL2Deposit
assertions.go:369: Messages: Expected to be online
assertions.go:369:
--- FAIL: TestL1ToL2Deposit (10.01s)
```
Root cause and fix.
Contributor guide
Research direction
Reproduce the CircleCI job and trace the initialization failures through op-devstack/dsl/eoa.go and op-devstack/dsl/el.go, then compare the callers in tests/base/faucet_test.go and tests/base/deposit/deposit_test.go. Determine why the faucet request returns 404 and why the service never becomes online; done means the initialization failures are fixed and these acceptance tests run reliably.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- ci-cd, testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100