Decentralized NAT traversal using nodes in the network

As invited by @vasco-santos in https://github.com/libp2p/js-libp2p/issues/870, I'm creating a more detailed overview of HoprConnect, an alternate transport module for js-libp2p handling churn and NAT traversal.

Disclaimer: parts of the documentation are taken from our own documentation and therefore slightly HOPR-flavoured.

Rationale

HoprConnect was created in the context of HOPR as js-libp2p-tcp as well as js-libp2p-webrtc-star did not support automatic NAT traversal or required external resources such as (external) STUN or (external) TURN and the final NAT traversal required some tweaks on the client software. The idea was to encapsulate most of the logic that is required to tunnel consumer routers in a transport module and work on higher-level mechanisms such as packet mixing.

Desired properties

• tunnel consumer routers, not just relay traffic over publicly available nodes
• no dependence on external resources such as STUN servers or TURN servers
• decentralized relay network, nodes can find neighbors in the network that help them tunneling NATs
• handling churn: nodes may come and leave
• compliance with js-libp2p Transport API specifications

Addressing

HoprConnect uses two kind of addresses:

• Direct addresses:
  /ip4/<IPv4 address>/tcp/<port>/p2p/<HOPR address>
/ip6/<IPv6 address>/tcp/<port>/p2p/<HOPR address>
  A node is available at the given IP address using the given TCP port and it is expected to talk to a node that has the HOPR address given through the Multiaddr. Direct connections using UDP or QUIC are not yet supported.
• Relay address
  /p2p/<HOPR relay address>/p2p-circuit/p2p/<HOPR address>
  A node is available by first establishing a connection to the relay node as given by the first HOPR address. The relay is then asked to establish a connection to the second HOPR address.

Socket interfaces

HoprConnect binds to a TCPv4 and a UDPv4 socket, the ports can and are intended to be the same.

UDPv4 is used exclusively for answering STUN request, which means that every node using HoprConnect is also a potential STUN server.

TCPv4 is used for everything else.

IPv6 is foreseen but not yet implemented.

Connection setup

Assume that A intends to talk to B and A knows a few direct address from B as well as some indirect addresses aka relay addresses.

A first tries to contact B using the direct addresses which can fail if the other node is living behind a NAT router. If this works, then the connection is kept.

Otherwise the node tries to connect to one of the given relays by using the indirect addresses. Once the connection to the relay is established, the node asks the relay to establish a connection to the final destination, B. The relay tries to contact the requested node and answers with OK if successful or FAIL_COULD_NOT_REACH_COUNTERPARTY if not accessible. If the destination could not be reached by the relay, the node tries a different relay and if there is none, the connection attempt is aborted.

Once the relayed connection is established, the node starts exchanging payload data with the destination. At the same time, both nodes, A and B initiate a WebRTC connection and check whether A and B can connect directly.
If a direct connection is possible, the relayed connection is transparently replaced by a direct connection.

Reconnects

Reconnects between direct connections such as TCP and WebRTC instances are handled automatically and mostly transparently by the operating system and WebRTC.

For relayed connections, this need to be handled explicitly because nodes do not get that kind of feedback from the other nodes automatically. More precisely, the node on one end of the relay stays unaware of happenings on the other end as long as the information is not actively forwarded.

HoprConnect implements this behavior by giving feedback to the sender of the message whether it has been successfully forwarded or not. If this message cannot be forwarded then the connection is paused until the node reconnects. Note that the relay does not cache the messages, it just tells the sender to stop sending and reject the reception.

The connection stays “half-open” until the node on the other side reconnects and thereby overwrites the existing connection. Once that happens, the relay injects a RECONNECT message into the message stream, notifying the other party about the necessity to restart the encryption layer.

Once the relayed connection is established, the both nodes do exactly the same as when establishing a "normal" connection: they start a WebRTC instance at both ends of the connection and checkout whether they can connect directly and transparently switch to a direct WebRTC connection if that is possible.

Bootstrapping

Once a node is started, it first tries to detect its own public IPv4 address by using any node in the network to answer its STUN request.

The following is WIP

Afterwards, it tries to connect to known relay nodes and announce to other nodes behind which nodes it is available.

WIP End

Comparison with other NAT traversal techniques

• normal TCP: no NAT Traversal
• using UPNP or NAT-PMP: depends on the router and some routers don't understand it
• relay everything: relay(s) become a single point of failure and subject of congestion, also bad in terms of privacy
• webRTC-star: needs external signalling servers as well as external TURN servers, different processes and addtional ports
• using a mixture of the aforementioned solutions: needs a lot of extra work when building decentralized software on the client

Potential browser-to-browser extension

The relay code is kept pretty agnostic where the connection comes from, which means that it can easily accept a HTTP(S) or even a WebSocket (Secure) stream and feed this stream into another stream on the other side of the relay. The missing part here is a browser implementation that establishes a relayed HTTP or WebSocket stream with one of the relay nodes and then transparently replace it with direct WebRTC connection if this is possible, otherwise it should keep the relayed connection.

The issue describes HoprConnect as an alternate js-libp2p transport and references the js-libp2p Transport API, NAT traversal, relay addressing, reconnects, and bootstrapping, but names no files or tests. Start by locating the relevant transport specifications and compare their required scope with the behaviors described here. Done would require an agreed specification or implementation plan for the decentralized transport, including its connection, relay, reconnect, and bootstrap behavior.