ethereum / ethereum/execution-specs
feat(consume): track any possible stats during test execution
- Dominant language
- Python
- Stars
- 1.2k
- Forks
- 505
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 116
Description
It could be useful to gather information about client behavior during simulator test execution, for example:
- Stats on client start-up (readiness) times.
- Number of retry requests against specific endpoints (see #1532).
- Stats on API request times, for example, for payload validation via `engineNewPayload` requests (the chart was from the [2935 `test_block_hashes_history_at_transition[fork_CancunToPragueAtTime15k-blockchain_test-blocks_before_fork_1-blocks_after_fork_257]`](https://eest.ethereum.org/main/tests/prague/eip2935_historical_block_hashes_from_state/test_block_hashes/test_block_hashes_history_at_transition/) test and revealed unexpected non-linear behavior in the time required due to a memory bug; that was very specific, but we could measure all requests and try to show variation over runs):

None of these behavior are specifically under test (timeouts lead to errors, of course); we'd need to find a way to make this data readily visible and perhaps add alerts for anomalies.
Contributor guide
Assessment
This issue has not been assessed yet.