ChainSafe / ChainSafe/gossamer

Introduce and implement `DhtEventStream`

Open
#4,621 1 comment 0 reactions 1 assignee Claimed by @freddyli7 View on GitHub
Dominant language
Go
Stars
454
Forks
144
PR merge metrics
No merged PRs in 30d

Description

## From design doc:

#### `DhtEventStream`

`DhtEventStream` is essentially a stream of item type [`DhtEvent`] enum. The enum is as follows:
```rust
pub enum DhtEvent {
/// The value was found.
ValueFound(PeerRecord),

/// The requested record has not been found in the DHT.
ValueNotFound(Key),

/// The record has been successfully inserted into the DHT.
ValuePut(Key),

/// An error has occurred while putting a record into the DHT.
ValuePutFailed(Key),

/// An error occured while registering as a content provider on the DHT.
StartProvidingFailed(Key),

/// The DHT received a put record request.
PutRecordRequest(Key, Vec, Option, Option),

/// The providers for [`Key`] were found.
ProvidersFound(Key, Vec),

/// The providers for [`Key`] were not found.
ProvidersNotFound(Key),
}
```

In Gossamer we currently use `go-libp2p-kad-dht` package for DHT functionality. `go-libp2p-kad-dht` does not currently emit any of these events. The `Worker` only handles cases of `ValueFound`, `ValueNotFound`, `ValuePut`, `ValuePutFailed`, and `PutRecordRequest`.

#### Implementing `DhtEventStream`

The rust `libp2p-kad` crate crate emits events of enum type [`KademliaEvent`](https://github.com/libp2p/rust-libp2p/blob/master/protocols/kad/src/behaviour.rs#L2745). The ones that need to be implemented in the Go Kademlia DHT library are `KademliaEvent::OutboundQueryProgressed` and `KademliaEvent::InboundRequest`.

##### `KademliaEvent::InboundRequest`
`InboundRequest` has a `request` attribute which is of type [`InboundRequest`](https://github.com/libp2p/rust-libp2p/blob/master/protocols/kad/src/behaviour.rs#L2854). If the request is of type [`PutRecord`](https://github.com/libp2p/rust-libp2p/blob/master/protocols/kad/src/behaviour.rs#L2878) we should be emitting an event that we can translate to `DhtEvent::PutRecordRequest`.

In `go-libp2p-kad-dht` it is unclear if we are able to listen on events that are inbound (aka come from other nodes). We will need to investigate into the codebase to see if there are current events that can be listened on to achieve this functionality.

##### `KademliaEvent::OutboundQueryProgressed`
For `OutboundQueryProgressed` there is a `result` attribute of type [`QueryResult`](https://github.com/libp2p/rust-libp2p/blob/master/protocols/kad/src/behaviour.rs#L2887). If the result is of variant type `QueryResult::GetRecord` this signals that an outbound request has been made to the DHT. `QueryResult::GetRecord` is of type `GetRecordResult` which is a result type `Result`. `GetRecordOK` is as follows:
```rust
/// The successful result of [`Behaviour::get_record`].
pub enum GetRecordOk {
FoundRecord(PeerRecord),
FinishedWithNoAdditionalRecord {
/// If caching is enabled, these are the peers closest
/// _to the record key_ (not the local node) that were queried but
/// did not return the record, sorted by distance to the record key
/// from closest to farthest. How many of these are tracked is configured
/// by [`Config::set_caching`].
///
/// Writing back the cache at these peers is a manual operation.
/// ie. you may wish to use these candidates with [`Behaviour::put_record_to`]
/// after selecting one of the returned records.
cache_candidates: BTreeMap,
},
}
```
There be an emitted event of `DhtEvent::ValueFound` whenever `GetRecordOk::FoundRecord` is the result. If there's an error `GetRecordError`, a `DhtEvent::ValueNotFound` event should be sent over the `DhtEventStream`.

If `OutboundQueryProgressed` result attribute is of type `QueryResult::PutRecord`, we should be emitting a `DhtEvent::ValuePut` event if it was succsesful, and a`DhtEvent::ValuePutFailed` event if it failed.

In `go-libp2p-kad-dht` both the `GetValue` and `PutValue` functions are synchronous calls that take in a supplied `context.Context` for cancellation. We should be able to emit these events in a type that wraps `IpfsDHT` by spawning goroutines to emit the `DhtEvent` variants.

## Acceptance Criteria
- Implement `DhtEventStream`
- Unit tests

[`Worker`]:https://github.com/paritytech/polkadot-sdk/blob/4b054c60b1641612ef0a76dcc75eed5dd23a18cf/substrate/client/authority-discovery/src/worker.rs#L233
[`Service`]:https://github.com/paritytech/polkadot-sdk/blob/414a8fc2eda3bb72e30cefdba628cf6c361cd6e1/substrate/client/authority-discovery/src/service.rs#L34
[`Role`]:https://github.com/paritytech/polkadot-sdk/blob/4b054c60b1641612ef0a76dcc75eed5dd23a18cf/substrate/client/authority-discovery/src/worker.rs#L86
[`AuthorityDiscovery`]:https://github.com/paritytech/polkadot-sdk/blob/4b054c60b1641612ef0a76dcc75eed5dd23a18cf/substrate/client/authority-discovery/src/worker.rs#L205
[`ProvideRuntimeApi`]:https://github.com/paritytech/polkadot-sdk/blob/c5444f381fdba68aa9cb73b39cc63f34604da156/substrate/primitives/api/src/lib.rs#L750
[`HeaderBackend`]:https://github.com/paritytech/polkadot-sdk/blob/fdb4554e26ebdd4d729158501a3ddb3c6ebdfb6f/substrate/primitives/blockchain/src/backend.rs#L37
[`NetworkProvider`]:https://github.com/paritytech/polkadot-sdk/blob/4b054c60b1641612ef0a76dcc75eed5dd23a18cf/substrate/client/authority-discovery/src/worker.rs#L1029
[`DhtEvent`]:https://github.com/paritytech/polkadot-sdk/blob/4b054c60b1641612ef0a76dcc75eed5dd23a18cf/substrate/client/network/src/event.rs#L35
[`Service`]:https://github.com/paritytech/polkadot-sdk/blob/414a8fc2eda3bb72e30cefdba628cf6c361cd6e1/substrate/client/authority-discovery/src/service.rs#L34
[`ServicetoWorkerMessage`]:https://github.com/paritytech/polkadot-sdk/blob/80616f6d03661106326b621e9cc3ee1d2fa283ed/substrate/client/authority-discovery/src/lib.rs#L168

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.