hiero-ledger / hiero-ledger/hiero-consensus-node

Pull-based reconnect improvements

Open
#23,970 3 comments 0 reactions 3 assignees Claimed by @akugal View on GitHub
Epic Performance Platform Platform Reconnect
Dominant language
Java
Stars
406
Forks
226
Avg merge
3d 4h
Merged PRs (30d)
210

Description

A few things have to be improved before pull-based reconnect mode can be enabled by default:

* `PullVirtualTreeRequest` / `PullVirtualTreeResponse`: binary serialization with `SerializableDataInput/OutputStream` should be changed to protobuf serialization. It would let us get rid of `SelfSerializable`, class IDs, and so on
* `AsyncInputStream` and `AsyncOutputStream` lifecycle. Currently, it's quite tangled. Async input stream is finished, once it receives "-1" as the message length. So it must be the output stream on the other side to send it. Output streams send "-1" when they are done processing all messages, but this is not enough: there may not be messages in the queue, but they may appear later. That's why `AsyncOutputStream` has a field `alive`, which is checked at every loop iteration. This `alive` is set differently on the learner and on the teacher sides. This is all very complicated and confusing
* Current code in `TeachingSynchronizer`, `LearningSynchronizer`, and other related classes works for both push- and pull-based reconnects. When push-based implementation is dropped, this code can be simplified a lot
* `TreeView`, `LearnerTreeView`, `TeacherTreeView`, and related interfaces can be simplified, too, or even removed. Many methods in these interfaces are legacy and only applicable to push-based reconnects
* Reconnect stats need a cleanup. The class is `ReconnectMapMetrics`, it tracks many different stats, but some of them are not applicable to pull-based implementations
* Performance. Pull-based mode(s) should be faster than push-based. This is often not the case today
* Exception / error handling should be improved. When an exception is thrown today, the stack trace is nearly impossible to inspect

I'm not sure if this ticket should be an epic or not. Some of the items above can be done independently, some are related.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.