lambdaclass / lambdaclass/lambda_ethereum_consensus

Epoch processing performance optimizations

Open
#1,399 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

perf
Dominant language
Elixir
Stars
109
Forks
41
PR merge metrics
No merged PRs in 30d

Description

After some work on reducing slot processing time, which greatly enhance block processing we still have a big bottleneck on every epoch processing. After some measures I was able to pinpoint the specific part of the code were 95% of the time was taken in the epoch processing execution and it was in a particular zip operation under rewards and penalties calculation as show by the following logs:

Image

This could be found here, but the line was modified to an Enum.zip instead of Stream, to be able to measure the bottleneck.

Something more caught my eyes when following the logs, there were 3 things that could be potential candidates for performance optimizations:

  1. I'm not completely sure but as far as the log shows, epoch processing occurs more than once within 3/4 blocks of difference instead of happening once every 32 blocks, there might be something involved related to the triggering of the epoch processing
  2. committee prefetch happens more than once nearly at the dame time due to the cache not storing the previous process calculating it in the meantime
  3. slot processing recalculates block and state root due to them not being available which apparently just happens near the epoch processing as shown in these logs

Image

This could easily become an umbrella Issue with sub task for tackling the aforementioned discoveries one at a time.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in lib/lambda_ethereum_consensus/state_transition/epoch_processing.ex around line 415 and use the reported profiling logs to examine the rewards and penalties zip operation. Then investigate the reported repeated epoch processing, committee prefetching, and block/state root recalculation; done means the selected bottleneck or subtask has a measured performance improvement.

Written by the indexing model from the issue text.

Assessment

Tech stack
elixir
Domain
backend, performance
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.