Proposal: a radically simpler Peerstore

Observation: The current Peerstore does a lot of things, but it doesn’t any of them well.

Problems:

• It’s a giant interface (32 methods in total (!!!), plus 2 for the CertifiedAddressBook).
• Entries are GC’ed when the peer disconnects.
• It’s trying to offer applications a range of options to store peer-related data, without offering options to persist this data beyond the time a peer disconnects.
• It doesn’t keep track of any metrics (e.g. regarding peer addresses stored).
• SupportsProtocol doesn’t distinguish between Identify having completed and not, leading to complexity in applications (example).

For example, this shows up:

• In QUIC / TLS: it can’t be used to store session tickets for session resumption / 0-RTT.
• In Kademlia: it can’t be used to store peer / provider records.

This is a proposal to radically shrink the peerstore, such that it offers only what is needed to dial connections. Applications running on top of QUIC, and even transports like QUIC that want to use 0-RTT, will need to come up with their own data structures to store peer-related data. This is inevitable, since different applications will have different needs regarding persisting data when a peer disconnects, and different heuristics for limiting the size of these data structures.

In particular, we’ll get rid of the following components:

1. PeerMetadata: Only used to save the agent / protocol version sent in Identify. There’s no real reason to save that information in the first place.
2. Metrics: Only used to store RTT statistics. Arguably, that’s not a per-peer, but a per-connection (or rather, per path, if dealing with multipath transports) property. We should consider exposing the RTT on the Connection interface.
3. ProtoBook: Used to store the protocols supported by this peer (as learned via Identify). Takes up huge amounts of storage, for little benefit. Protocols that need a list of peers supporting their protocol can easily subscribe to the respective EventBus notification. [^1]

The KeyBook will be radically simplified to only allow storing of the public key.

As a result, this leaves us with a peer store that only stores:

1. addresses
2. public keys
3. certified peer records (in their serialized form)

It basically becomes an address book, and maybe we should honor that fact by renaming it accordingly.

The following changes will make it a lot more smarter and more useful:

• Don’t GC on disconnect. Keep a scoring function of how valuable an entry is, and GC based on that. An entry is valuable, if we’ve connected to a peer multiple times, the node has been online for a long time, or an application tells us that a peer is (based on some application-defined metric, not sure yet how exactly this would work).
• Keep track if an address is received securely (e.g. in a signed peer record, on an encrypted connection to that peer, or if we dialed / accepted a connection on that address and completed the handshake). If so, we could ignore all unverified addresses.
• Keep stats on how well addresses work in practice, by tracking attempted and successful connection attempts on a per-address basis. This can be fed into the dialing logic to speed up future dials.

We could consider removing the Peerstore interface. This is (mostly) a libp2p-internal component. In particular, it does not make sense to implement a datastore-backed version of it (it might however be interesting to be able to serialize the current state, so it can be restored easily).

[^1]: Note that this breaks an optimization for Host.NewStream, which currently picks the first supported protocol from a list of protocols. This is fine, since 1. worst-case, this results in an extra round-trip, 2. it is a rare use-case and 3. the application can track which protocols a peer speaks and then open a stream with just that protocol, speeding up things by 1 RTT compared to the status quo.

Start by reviewing the Peerstore interface and its KeyBook, PeerMetadata, Metrics, and ProtoBook components, then inspect the Host.NewStream behavior and p2p/host/autorelay/relay_finder.go example referenced in the issue. Before implementation, establish the replacement scope and migration plan; done means the agreed peerstore redesign is implemented without the listed responsibilities.