haskell-distributed / haskell-distributed/distributed-process

Include outgoing connection events in receive queue

Open
#405 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

feature network-transport
Dominant language
Haskell
Stars
751
Forks
99
Avg merge
45m
Merged PRs (30d)
2

Description

This is an alternative to, and intended to solve the same problem as, issue haskell-distributed/distributed-process#406.

New Events

Three new event constructors are introduced:

data Event =
    ...
  | OutboundConnectionOpened ConnectionId Reliability EndPointAddress
  | OutboundConnectionClosed ConnectionId
  | OutboundConnectionSent ConnectionId [ByteString]

Or perhaps instead:

data Provenance = Local | Peer
  deriving (Show, Eq, Generic)

instance Binary Provenance

data Event =
    -- Replaces 'Received', which can now mean 'Sent'.
    Data Provenance ConnectionId [ByteString]
  | ConnectionOpened Provenance ConnectionId Reliability EndPointAddress
  | ConnectionClosed Provenance ConnectionId
  | ...
  deriving (Show, Eq, Generic)

pattern Received connid bss = Data Peer connid bss
pattern Sent connid bss = Data Local connid bss

and in any case:

data Connection = Connection {
    ...
  , connectionId :: ConnectionId
  }

Events with provenance Peer are the familiar incoming events, whereas events with provenance Local are generated in response to locally-initiated connection features: connect, send, and close.

Semantics of local-provenance Events

When connect ep address reliability hints succeeds, a ConnectionOpened Local connid reliability address must be posted exactly once to eps event queue, where connid is any suitable (unique) ConnectionId.

When close conn succeeds, a ConnectionClosed connid must be posted exactly once to the event queue of the EndPoint against which conn was created.

When send conn bs succeeds, a Data Local connid bs must be posted exactly once to the event queue of the EndPoint against which conn was created, even if the connection is not reliable (in which case the receiver may see different Data events than the sender for the same connection).
If the connection is ordered, then Data Local events must be posted in the same order that the peer will observe the Data Peer events corresponding to these sends (it's assumed the concrete transport is capable of doing this, else it shouldn't allow an ordered connection).
Regardless of reliability and ordering, a Data Local event must come after the ConnectionOpened for that connection, and before any event which terminates that connection (ConnectionClosed or error events which cause it to close).

Note that these rules imply that for self connections (from one EndPoint to itself), the EndPoint will see two of each posted event, one for each provenance (they are both Local and Peer).

The connectionId of a Connection must be unique among incoming and outgoing connections, but need not match the ConnectionId which the peer uses for that same connection.

It solves the problem brought up in haskell-distributed/distributed-process#406

There is now an ordering on outgoing connections. If we observe ErrorEvent (TransportError (EventConnectionLost addr) _) then every ConnectionOpened Local connid _ addr which is unmatched by a ConnectionClosed Local connid is known to be broken, and subsequent ConnectionOpened Local connid' _ addr are not broken by that error event.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading issue haskell-distributed/distributed-process#406 and comparing its problem with the two alternative Event designs described here. Then inspect the existing Event and Connection definitions and the connect, send, and close paths. Done means choosing a design and preserving the specified provenance, ordering, uniqueness, and exactly-once event semantics.

Written by the indexing model from the issue text.

Assessment

Tech stack
haskell
Domain
distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.