ethereum / ethereum/hive

Simulator attempts to get genesis block prematurely before Besu finished starting

Open
#802 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
523
Forks
410
Avg merge
2d 8h
Merged PRs (30d)
5

Description

This issue is discovered as part of the effort of fixing up the failed consensus tests on Besu hyperledger/besu#5689

**Description**

Occasionally, simulator fails to get genesis block in a test by showing the following error message:
`can't get genesis: Post "http://172.17.0.9:8545": dial tcp 172.17.0.9:8545: connect: connection refused`
This is due to api call was made while Besu was still initializing. This appears to happen at random, a failed test that has this error in current run may pass in the next run. I am unable to reproduce this issue on my local machine. The associated failed tests on [https://hivetests.ethdevops.io](https://hivetests.ethdevops.io) ran fine on my local.

**Example**

`pointMulAdd2_d31g0v0_London` on [Jul 6 run](https://hivetests.ethdevops.io/suite.html?suiteid=1688873269-72e5670d42e9d7bf0116da51fbf048e4.json&suitename=consensus#) [(log)](https://hivetests.ethdevops.io/viewer.html?suiteid=1688873269-72e5670d42e9d7bf0116da51fbf048e4.json&suitename=consensus&testid=82932&file=%2Fresults%2Fbesu%2Fclient-91ce8a32c9712129c004a16970c04d68dea51426f81af957c1e7607afa6b5f8e.log) failed but passed on [Jul 12 run](https://hivetests.ethdevops.io/suite.html?suiteid=1689456616-ca5e33a64c81ae6d20e92b9eba0b947b.json&suitename=consensus#) [(log)](https://hivetests.ethdevops.io/viewer.html?suiteid=1689456616-ca5e33a64c81ae6d20e92b9eba0b947b.json&suitename=consensus&testid=82565&file=%2Fresults%2Fbesu%2Fclient-94aab8e7b805ce9af025b161905a07b2244e624d99f417ca67f7d52ca05e8e67.log)
`envInfo_d3g0v0_Merge` passed on Jul 6 [(log)](https://hivetests.ethdevops.io/viewer.html?suiteid=1688873269-72e5670d42e9d7bf0116da51fbf048e4.json&suitename=consensus&testid=3397&file=%2Fresults%2Fbesu%2Fclient-6cdf5502868e12bdddc501f825c37af26c53190367a5ad8f91a07020ea37f23d.log) but failed on Jul 12 [(log)](https://hivetests.ethdevops.io/viewer.html?suiteid=1689456616-ca5e33a64c81ae6d20e92b9eba0b947b.json&suitename=consensus&testid=3425&file=%2Fresults%2Fbesu%2Fclient-ce79ae20c214574684d859e01bbc16dccca002164ae3e403dcf1ec710b688198.log)

Normally after Besu starts, there should be a message `Ethereum main loop is up.` indicating it is ready to receive rpc requests. This is shown on Jul 12 log, but Jul 6 log indicates Besu was abruptly stopped while it was still initializing.

**Speculation**

It could be [`StartClient()`](https://github.com/ethereum/hive/blob/c1d0d557f97bc7478c83724b4c8ba80eaa652e3b/simulators/ethereum/consensus/main.go#L474) did not correctly determine if Besu is ready to receive requests. The retry mechanism in [`getBlock()`](https://github.com/ethereum/hive/blob/c1d0d557f97bc7478c83724b4c8ba80eaa652e3b/simulators/ethereum/consensus/main.go#L569) did get triggered but failed on both first and second attempt and there lacked a waiting time between the two attempts

Contributor guide

No contributing guide indexed for this repository

Research direction

Start in simulators/ethereum/consensus/main.go at StartClient() and getBlock(), and review how readiness is detected from the “Ethereum main loop is up.” message. Run the affected consensus tests and inspect the retry behavior around the genesis-block request; done means the simulator does not request genesis before Besu accepts RPC requests and intermittent startup failures are avoided.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, go
Domain
devtools, testing-qa
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.