ethereum / ethereum/execution-specs

feat(execute): improve execute hive mode markers and session scope

Open
#1,550 3 comments 1 reaction 0 assignees View on GitHub
A-test-execute C-feat stale
Dominant language
Python
Stars
1.2k
Forks
505
Avg merge
2d 14h
Merged PRs (30d)
116

Description

### Description

Execute hive mode works well currrently, but can be improved in the future primarily to aid test fail debugging.

Following the comments on this hive PR: https://github.com/ethereum/hive/pull/1291

>
> About the -m get_blobs...
>
> Here's just one other small bit of relevant info... and that's that we only parametrize a simulator with the format if it's supported by the particular simulator (_supported_fixture_formats is defined in the simulator's local conftest.py):
>
> https://github.com/ethereum/execution-spec-tests/blob/020f728eefedc75a68e6fba6dbc65322e1d9f490/src/pytest_plugins/consume/consume.py#L411-L423
>
> This avoids having to add these command-line args, which is beneficial if we want to allow the user to specify other markers (as in https://github.com/ethereum/hive/pull/1248).

> Future improvements summarized below from my understanding:
> 1) Remove the marker usage -m from the dockerfile.
> 2) Replace this directly within EEST where we specify the test types we want to execute hive mode to run.
> 3) Run execute hive mode using pytests session level so we can easily filter tests within the hive ui like we do for consume!

To achieve this we must be able to run multiple tests against a single client instance within hive. This is currently a work in progress by @danceratopz for `consume engine` to allow us to run all of the engine tests significantly faster. Once the latter is completed we can move forward with the execute hive mode improvements.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.