Reduce the size of sessions objects
- Dominant language
- Python
- Stars
- 19
- Forks
- 15
- PR merge metrics
- No merged PRs in 30d
Description
## What was wrong?
The `Session` objects take up around 40k. Ideally we want to keep a cache of maybe 1k-10k of these. At the current size, this means 40-400mb of memory going towards these objects.
## How can it be fixed?
Sessions currently contain a mix of logic:
1. The logic for doing handshake negotiation
2. The logic for handling messages after handshake is complete.
We only need the handshake negotiation logic while the negotiation is happening. Afterwards, we only need a much smaller amount of the logic.
### Handshake Stuff
holding onto important packets that are only needed during the handshake
- https://github.com/ethereum/ddht/blob/fd5dd92933d336b49383808c2c78eb7e28159b0c/ddht/v5_1/session.py#L274
- https://github.com/ethereum/ddht/blob/fd5dd92933d336b49383808c2c78eb7e28159b0c/ddht/v5_1/session.py#L344
- https://github.com/ethereum/ddht/blob/fd5dd92933d336b49383808c2c78eb7e28159b0c/ddht/v5_1/session.py#L626-L632
the envelope buffers that are only used during handshake negotiation
- https://github.com/ethereum/ddht/blob/fd5dd92933d336b49383808c2c78eb7e28159b0c/ddht/v5_1/session.py#L472-L475
The `ENRDatabase` is only needed during handshake
- https://github.com/ethereum/ddht/blob/fd5dd92933d336b49383808c2c78eb7e28159b0c/ddht/v5_1/session.py#L78
There is almost definitely a *lot* more stuff that is only needed during the handshake
### Full Session Stuff
Once a handshake has been completed, the number of possible code paths drop significantly. We only have to deal with `MessagePacket` types. All other packet types can be discarded.
It's unclear exactly how this should work, but...
- An **elegant** version of this might be a function which takes a set of `SessionKeys` and a few other things and does what the current code paths do for inbound messages.
- A **simple** version might be a new `Session` class that we *promote* the current session objects to once they have completed the handshake negotiation.
- A **pragmatic** approach might be to do some clever deletions of the various *things* that we no longer need once the handshake completes.
I think the best way to figure this out would be to first figure out where the current 40k comes from. Then, explore the *pragmatic* approach to see if we can't get the size down by simply deleting things we don't need. Afterwards we should have enough information to know whether we need to actually change the session model.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.