ethereum / ethereum/execution-specs

feat(consume): track any possible stats during test execution

Open
#1,544 1 comment 0 reactions 0 assignees View on GitHub
A-test-consume C-feat P-low stale
Dominant language
Python
Stars
1.2k
Forks
505
Avg merge
2d 14h
Merged PRs (30d)
116

Description

It could be useful to gather information about client behavior during simulator test execution, for example:
- Stats on client start-up (readiness) times.
- Number of retry requests against specific endpoints (see #1532).
- Stats on API request times, for example, for payload validation via `engineNewPayload` requests (the chart was from the [2935 `test_block_hashes_history_at_transition[fork_CancunToPragueAtTime15k-blockchain_test-blocks_before_fork_1-blocks_after_fork_257]`](https://eest.ethereum.org/main/tests/prague/eip2935_historical_block_hashes_from_state/test_block_hashes/test_block_hashes_history_at_transition/) test and revealed unexpected non-linear behavior in the time required due to a memory bug; that was very specific, but we could measure all requests and try to show variation over runs):
![Image](https://github.com/user-attachments/assets/e615af91-0ed3-49db-9e3b-c5fa38c3ad5e)

None of these behavior are specifically under test (timeouts lead to errors, of course); we'd need to find a way to make this data readily visible and perhaps add alerts for anomalies.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.