lambdaclass / lambdaclass/lambda_ethereum_consensus
Epoch processing performance optimizations
Nobody has claimed this yet.
- Dominant language
- Elixir
- Stars
- 109
- Forks
- 41
- PR merge metrics
- No merged PRs in 30d
Description
After some work on reducing slot processing time, which greatly enhance block processing we still have a big bottleneck on every epoch processing. After some measures I was able to pinpoint the specific part of the code were 95% of the time was taken in the epoch processing execution and it was in a particular zip operation under rewards and penalties calculation as show by the following logs:
This could be found here, but the line was modified to an Enum.zip instead of Stream, to be able to measure the bottleneck.
Something more caught my eyes when following the logs, there were 3 things that could be potential candidates for performance optimizations:
- I'm not completely sure but as far as the log shows, epoch processing occurs more than once within 3/4 blocks of difference instead of happening once every 32 blocks, there might be something involved related to the triggering of the epoch processing
- committee prefetch happens more than once nearly at the dame time due to the cache not storing the previous process calculating it in the meantime
- slot processing recalculates block and state root due to them not being available which apparently just happens near the epoch processing as shown in these logs
This could easily become an umbrella Issue with sub task for tackling the aforementioned discoveries one at a time.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in lib/lambda_ethereum_consensus/state_transition/epoch_processing.ex around line 415 and use the reported profiling logs to examine the rewards and penalties zip operation. Then investigate the reported repeated epoch processing, committee prefetching, and block/state root recalculation; done means the selected bottleneck or subtask has a measured performance improvement.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- elixir
- Domain
- backend, performance
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100