ethereum / ethereum/ddht

Reduce the size of sessions objects

Open
#210 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
19
Forks
15
PR merge metrics
No merged PRs in 30d

Description

## What was wrong?

The `Session` objects take up around 40k. Ideally we want to keep a cache of maybe 1k-10k of these. At the current size, this means 40-400mb of memory going towards these objects.

## How can it be fixed?

Sessions currently contain a mix of logic:

1. The logic for doing handshake negotiation
2. The logic for handling messages after handshake is complete.

We only need the handshake negotiation logic while the negotiation is happening. Afterwards, we only need a much smaller amount of the logic.

### Handshake Stuff

holding onto important packets that are only needed during the handshake

- https://github.com/ethereum/ddht/blob/fd5dd92933d336b49383808c2c78eb7e28159b0c/ddht/v5_1/session.py#L274
- https://github.com/ethereum/ddht/blob/fd5dd92933d336b49383808c2c78eb7e28159b0c/ddht/v5_1/session.py#L344
- https://github.com/ethereum/ddht/blob/fd5dd92933d336b49383808c2c78eb7e28159b0c/ddht/v5_1/session.py#L626-L632

the envelope buffers that are only used during handshake negotiation

- https://github.com/ethereum/ddht/blob/fd5dd92933d336b49383808c2c78eb7e28159b0c/ddht/v5_1/session.py#L472-L475

The `ENRDatabase` is only needed during handshake

- https://github.com/ethereum/ddht/blob/fd5dd92933d336b49383808c2c78eb7e28159b0c/ddht/v5_1/session.py#L78

There is almost definitely a *lot* more stuff that is only needed during the handshake

### Full Session Stuff

Once a handshake has been completed, the number of possible code paths drop significantly. We only have to deal with `MessagePacket` types. All other packet types can be discarded.

It's unclear exactly how this should work, but...

- An **elegant** version of this might be a function which takes a set of `SessionKeys` and a few other things and does what the current code paths do for inbound messages.
- A **simple** version might be a new `Session` class that we *promote* the current session objects to once they have completed the handshake negotiation.
- A **pragmatic** approach might be to do some clever deletions of the various *things* that we no longer need once the handshake completes.

I think the best way to figure this out would be to first figure out where the current 40k comes from. Then, explore the *pragmatic* approach to see if we can't get the size down by simply deleting things we don't need. Afterwards we should have enough information to know whether we need to actually change the session model.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.