IntersectMBO / IntersectMBO/ouroboros-consensus

Leios: fetch buffers pin the last request's transactions for the life of the connection

Open
#2,297 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Haskell
Stars
67
Forks
43
Avg merge
5d 13h
Merged PRs (30d)
43

Description

`newLeiosFetchContext` allocates [two scratch buffers sized `maxTxsPerEb`](https://github.com/IntersectMBO/ouroboros-consensus/blob/b56977baae0740f563060a8a9171c78be865b357/ouroboros-consensus/src/ouroboros-consensus/LeiosDemoLogic.hs#L239-L240), one pair per LeiosFetch responder, so one pair per hot peer.
Nothing clears them between requests, so each connection pins its largest served request until it closes.

The serving code copies out the used prefix and leaves the rest ([`LeiosDemoLogic.hs:279`](https://github.com/IntersectMBO/ouroboros-consensus/blob/b56977baae0740f563060a8a9171c78be865b357/ouroboros-consensus/src/ouroboros-consensus/LeiosDemoLogic.hs#L279), [`:316`](https://github.com/IntersectMBO/ouroboros-consensus/blob/b56977baae0740f563060a8a9171c78be865b357/ouroboros-consensus/src/ouroboros-consensus/LeiosDemoLogic.hs#L316)):

```haskell
v <- V.freeze $ MV.slice 0 n buf -- copies 0..n-1; buf keeps all its pointers
```

`MV.slice` is a view.
`V.freeze` copies.
So `buf` still holds every pointer it wrote, including those above the current `n`.

Both buffers are boxed ([`:50-51`](https://github.com/IntersectMBO/ouroboros-consensus/blob/b56977baae0740f563060a8a9171c78be865b357/ouroboros-consensus/src/ouroboros-consensus/LeiosDemoLogic.hs#L50-L51), [`:224-225`](https://github.com/IntersectMBO/ouroboros-consensus/blob/b56977baae0740f563060a8a9171c78be865b357/ouroboros-consensus/src/ouroboros-consensus/LeiosDemoLogic.hs#L224-L225)).
`Data.Vector.Strict.Mutable` forces elements to WHNF and keeps a pointer array, and `TxHash` wraps a `ByteString`, so an unboxed vector is not an option here.

What each one pins:

- `leiosEbBuffer :: MVector s (TxHash, BytesSize)`, up to `maxTxsPerEb` boxed pairs, each holding a 32-byte `ByteString`.
- `leiosEbTxsBuffer :: MVector s LeiosTx`, and [`newtype LeiosTx = MkLeiosTx ByteString`](https://github.com/IntersectMBO/ouroboros-consensus/blob/b56977baae0740f563060a8a9171c78be865b357/ouroboros-consensus/src/ouroboros-consensus/LeiosDemoTypes.hs#L969), so each live slot pins a whole transaction's CBOR.

One context is created per LeiosFetch responder instance in [`hLeiosFetchServer`](https://github.com/IntersectMBO/ouroboros-consensus/blob/b56977baae0740f563060a8a9171c78be865b357/ouroboros-consensus-diffusion/src/ouroboros-consensus-diffusion/Ouroboros/Consensus/Network/NodeToNode.hs#L710), registered under `withHot` with `StartOnDemand`.
The client path allocates none.
So N hot peers means N contexts.

The pointer array alone is 2 x 71428 x 8 = 1.09 MiB per context, up from 217 KiB at `maxTxsPerEb` = 13888.
At 50 hot peers that is 54.5 MiB against 10.6 MiB.
Retained payloads sit on top of that.

On the honest path the transaction buffer stays near [`maxRequestBytesSize` = 500 kB](https://github.com/IntersectMBO/ouroboros-consensus/blob/b56977baae0740f563060a8a9171c78be865b357/ouroboros-consensus/src/ouroboros-consensus/LeiosDemoTypes.hs#L933), because the client partitions its requests by that bound.
The server applies no byte cap of its own, and [`msgLeiosBlockTxsRequest` validates bitmap shape only](https://github.com/IntersectMBO/ouroboros-consensus/blob/b56977baae0740f563060a8a9171c78be865b357/ouroboros-consensus/src/ouroboros-consensus/LeiosDemoLogic.hs#L293-L299).
So a peer that bitmaps a whole EB pins its whole closure until the connection closes.

This is not new.
The pattern is byte-identical before #2280.
That PR raises `maxTxsPerEb` from 13888 to 71428, so both ceilings move by 5.14x.

An `MV.set` over the tail after the freeze would drop the references.
`unsafeFreeze` plus a fresh buffer per request is the other shape, at the cost of one allocation per request.

Contributor guide

Open the contributing guide

Research direction

Read the buffer allocation and serving paths in ouroboros-consensus/src/ouroboros-consensus/LeiosDemoLogic.hs, then trace context creation through hLeiosFetchServer in ouroboros-consensus-diffusion/.../NodeToNode.hs. Verify how requests reuse the boxed buffers and how LeiosTx is represented in LeiosDemoTypes.hs. Done means completed requests no longer retain pointers or payloads from larger prior requests, without changing the returned response.

Written by the indexing model from the issue text.

Assessment

Tech stack
haskell
Domain
backend, performance
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
74/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.