dropLowPriority() doesn't reclaim state control parts, and queued subscriptions grow unbounded
@lucassaldanha is already working on this.
Since Aug 14, 2026.
- Dominant language
- Kotlin
- Stars
- 366
- Forks
- 85
- Avg merge
- 2d 39m
- Merged PRs (30d)
- 6
Description
Found this while soak testing 1.3.6-RC2 with Teku on a hoodi node. Not urgent, but I think there are two related problems with the outbound parts queue.
dropLowPriority() only frees BULK parts
DefaultGossipRpcPartsQueue.dropLowPriority() drops priorityPartLists[BULK_PRIORITY], so publish, ihave and iwant. Subscription, graft and prune parts live in STATE_CONTROL_PRIORITY and never get dropped.
Those parts still count towards estimateMaxSerializedSize(), which is what GossipRouter.trackSlowPeers() compares against slowPeerPendingBytesThreshold. So for a peer whose queue is mostly state control parts, the queue stays above the threshold, notifySlowPeer fires every slowPeerHeartbeatThreshold heartbeats, trimOutboundQueue drops the bulk parts (possibly none), and the queue never drops back under the threshold. The bound added in 5a68b18a doesn't cover those parts.
Queued subscriptions keep growing
addSubscription appends a new part every time, so subscribe/unsubscribe churn on the same topic just piles up. Teku churns attestation subnet subscriptions every epoch, so there's a steady stream of them. Only the final state per topic actually matters on the wire, so I'd expect the queue to collapse them rather than keep every transition. Keying subscriptions by topic and overwriting the status would bound this to the number of topics.
Numbers from heap histograms on the node (single Teku process, ~200 peers, process uptime in the header):
24.4h 28.6h 34.6h 48.2h
SubscriptionPart 213,214 241,118 314,541 444,718
IHavePart 650,983 782,518 787,547 674,320
TTLSeenCache$TimedMessage 3,150,008 3,243,836 2,905,139 3,075,955
WBytes 1,691,278 1,815,221 1,629,733 1,674,305
SubscriptionPart is the only one that just goes up, roughly 9.7k/hour. Everything else oscillates with gossip load. That's about 2,200 per peer at the moment so it's a long way off the 4 MiB threshold.
Nothing else looks off on the node. Post GC old gen floor is flat over 48h, direct buffers flat at ~230MB, and one DroppedRpcPartsException in two days, which is the slow peer path working as intended.
Both of these come from the size aware batching in 02410783 rather than the threshold default in 5a68b18a. Before batching, takeMerged() cleared the whole queue on every flush so nothing could be retained.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.