Coupling between `transaction` and `chainhead` functions

Problem Statement

Currently, the only way to send a transaction is through transaction_unstable_submitAndWatch. This method undertakes several tasks:

• Validates the transaction against the latest finalized block.
• Broadcasts the transaction.
• Informs the consumer about the transaction's presence in the best and/or finalized block.

Despite its comprehensive functionalities, a significant challenge arises due to its interactions with another function: chaihead_unstable_follow. This function, belonging to a different group, provides crucial information about the latest finalized block, which is indispensable for creating valid transactions.

However, the inherent separation of these function groups and the lack of synchronization mechanisms pose risks:

1. Lack of Synchronization: If the transaction function lags behind the chainhead, it could result in users being unable to send transactions altogether.

2. Delayed Events: Events from the transaction function might be delayed or misaligned with the actual block status, rendering them ineffective for many applications. In such cases, users are forced to fetch block bodies manually and verify transaction presence, adding unnecessary complexity and duplication of efforts.

In essence, while the chainhead group of functions is properly isolated from other functionalities, the same can't be said for the transaction group. Its dependencies on the chainhead group bring up synchronization and information consistency concerns, which are pivotal for seamless transaction handling.

Proposed Solutions and Concerns

1. Tight Coupling of Function Groups

• Description: Assume that the transaction group of functions is intrinsically linked to the chainhead group. Hence, if transaction is available, so is chainhead. This means that transaction would internally use chainhead.
• Advantages: Straightforward approach; ensures that users would always be able to send transactions and also ordered message arrival.
• Drawbacks: Challenges the very essence of function separation for load-balancers; cannot guarantee the presence of both function groups as per the spec.

2. Decoupling transaction from chainhead

• Description: The only role for the transaction group would be broadcasting a transaction. This would shift validation and block body transaction tracking responsibilities to the client.
• Advantages: Simplicity; forward compatibility due to its low-level nature.
• Drawbacks: Heavier responsibility on the client side; potential performance concerns; the API becomes challenging for users less acquainted with chainhead.

3. Merging Function Groups

• Description: Combine the two groups of functions (transaction and chainhead) into a single group, indicating their close interdependency.
• Advantages: Fewer events and edge cases.
• Drawbacks: Nodes may want to avoid transaction support. Allowing only a subset of functions in a group is currently not possible and also undesirable.

4. Superset Approach with transaction

• Description: Allow the transaction group of functions to encompass the responsibilities of the chainhead group, effectively making transaction a supergroup. Same solution as before, but leaving chainhead untouched.
• Advantages: Workaround to the function subset challenge.
• Drawbacks: Duplication.

I understand that these solutions aren't exhaustive and hope they provide insight into the possible ways forward. As a spec consumer, I find it crucial not only to highlight issues but also to offer potential resolutions. Eager to hear your insights on this.

cc: @tomaka