Dynamic Relay Configuration Proposal

# Dynamic Relay Configuration Proposal

This proposal summarises our take on[ Support Validator Account Level Relay Choice](https://github.com/flashbots/mev-boost/issues/154) and [Validator Level Relay Configuration](https://github.com/flashbots/mev-boost/issues/416) as part of ConsenSys Codefi team responding to the features our customers requested.

## Requirements

Many stakers run multiple keys on the same validator/consensus layer client. **MEV-Boost**, as currently implemented, requires all keys on the same node to point at the same set of relays.

The objective of this proposal is to allow stakers to specify different relays for different validators without having to increase their consensus layer client population:

## Acceptance Criteria

- Different validator keys on the same validator client can point to different sets of relays
- There is no compromise on security or introduction of a new attack surface
- Minimal impact on the current latency sensitive operations of **MEV-Boost**
- The implementation is optional - users can continue to use **MEV-Boost** as they do today
- Changes to the relay set of a particular validator are adopted quickly by the infrastructure with no disruption (e.g.: restarting **MEV-Boost**)

## Definitions

**Relay Configuration Provide**r (**RCP**) holds information about validators and their corresponding relays. For example it may be as simple as a file or a remote API endpoint which returns a [Proposer Config](https://docs.teku.consensys.net/HowTo/Configure/Proposer-Configuration) - a JSON discussed in [Support validator account level relay choice](https://github.com/flashbots/mev-boost/issues/154) issue.

**Relay Configuration Manager (RCM)** manages relays and their configurations. It fetches the configuration from an **RCP**, validates it and then populates an in memory **Relay Registry** which is basically a map of all the registered validators and their corresponding relay configs. The **Relay Registry** is then used to lookup relays for a given validator when requested.

## Suggested Changes

We change **MEV-Boost** to:

1. Support a command line/env config flag to use an **RCM**.
2. Call the **RCM** to fetch all the relays for all the managed validators at startup.
3. Query the **RCM** to get relays for a given validator on every **MEV-Boost** API call.

When **RCM** is created at service startup it will try to fetch the configuration from an **RCP**. If this operation fails, the service will refuse to fast.

Then a background job is started to periodically synchronise the configuration. The job is optional and enabled only when **proposer-config-refresh-enabled** param is set to true. It will run every 3.2 mins (half an epoch). It is important to note that the **Relay Registry** won’t be updated if the configuration is malformed or invalid, in which case the previous valid configuration is used.

**MEV-Boost** then uses **RCM** to lookup information about relays for a given validator on every request to any endpoint (register validator, get header, get payload) via API.

## Schema

Given the expectation to support multiple configuration sources, the implementation should align on a common schema. The schema we follow is the one discussed in the [proposed changes](https://github.com/flashbots/mev-boost/issues/154) and already used by **Teku** and available via [**validators-proposer-config**](https://docs.teku.consensys.net/HowTo/Configure/Proposer-Configuration) option. The intention is to allow specifying a set of specific relays per validator and otherwise fallback to a common set of relays.

The **proposer_config** section contains the mapping from validator public key to the fee recipient and builder config. Inside the **builder** section, there is a mandatory **enabled** field. If **true**, the **relays** should be used by **MEV-boost**. If **false**, then **MEV-Boost** should ignore this key (and skip any incoming requests for this validator key).

If a validator key is not present inside the **proposer_config**, then the configuration inside **default_config** should be applied.

## F.A.Q

### Are the proposed changes backward compatible?

Yes they are. Using the newly introduced features is optional. At the moment we suggest a binary choice: either to use **RCM** or to use **MEV-Boost** in the conventional way.

### What RCPs are supported?

At the moment we have a file-based **RCP** available via **MEV-Boos**t’s **proposer-config-file** option and an API-based **RCP** configured via **proposer-config-url**.

### What happens when the Relay Configuration Provider is down?

When **RCM** is used then:

- If an **RCP** is down when **MEV-Boost** starts, then the service will fail fast with an error and refuse to start.
- If an **RCP** is down when **MEV-Boost** tries to synchronise the configuration, then the error is logged and the previously valid configuration will be used until the synchronisation is successful.

### What happens if the config is changed after a validator has been registered?

The changes will take effect in the next epoch.

### Should remote RCP be exposed to the internet?

When using a remote RCP we strongly encourage running it in a trusted DMZ, preferably in the same network as mev-boost. We don’t expect it to be publicly available.

## Test cases

### Valid Cases

| Proposer key | Proposer Config Builder | Proposer Relays | Default Config Builder | Default Relays | Result |
| ------------ | ----------------------- | --------------- | ---------------------- | -------------- | --------------------------- |
| Not Present | n/a | n/a | FALSE | EMPTY | No MEV used, EL block built |
| Not Present | n/a | n/a | TRUE | Filled | Use default relays |
| Present | NULL | n/a | FALSE | EMPTY | No MEV used, EL block built |
| Present | FALSE | Empty | FALSE | EMPTY | No MEV used, EL block built |
| Present | NULL | n/a | TRUE | Filled | Use default relays |
| Present | FALSE | Empty | TRUE | Filled | No MEV used, EL block built |
| Present | TRUE | Filled | TRUE | Filled | Use the proposer relays |
| Present | TRUE | Filled | FALSE | EMPTY | Use the proposer relays |

### Invalid cases

| Proposer key | Proposer Config Builder | Proposer Relays | Default Config Builder | Default Relays | Result |
| ------------ | ----------------------- | --------------- | ---------------------- | -------------- | ------ |
| Present | TRUE | Empty | \- | \- | Error |
| \- | \- | \- | TRUE | Empty | Error |
| \- | \- | \- | NULL/FALSE | \- | Ignore |
| \- | \- | \- | FALSE | Filled | Ignore |

### Benchmarks

```
goos: darwin
goarch: arm64
pkg: github.com/flashbots/mev-boost/server
```

Seconds per operation:

| Test | Before | After | Diff |
| ------------------- | ------------- | ------------ | ------------------------ |
| RegisterValidator-8 | 108.0µ ± 11 | 134.6µ ± 12% | \+24.63% (p=0.000 n=10 |
| GetHeader-8 | 1.762m ± 88% | 1.792m ± 2% | ~ (p=0.579 n=10) |
| GetPayload-8 | 1651.4µ ± 7% | 515.4µ ± 19% | \-68.79% (p=0.000 n=10) |
| geomean | 679.8µ | 499.1µ | \-26.59% |

Bytes per operation:

| Test | Before | After | Diff |
| ------------------- | ------------- | ------------ | ------------------------ |
| RegisterValidator-8 | 73.96Ki ± 1% | 118.86Ki ± 0% | \+60.71% (p=0.000 n=10) |
| GetHeader-8 | 180.1Ki ± 1% | 225.8Ki ± 0% | \+25.39% (p=0.000 n=10) |
| GetPayload-8 | 711.4Ki ± 1% | 221.1Ki ± 2% | \-68.92% (p=0.000 n=10) |
| geomean | 211.6Ki | 181.1Ki | \-14.44% |

Allocations per operation:

| Test | Before | After | Diff |
| ------------------- | ------------- | ------------ | ------------------------ |
| RegisterValidator-8 | 921.0 ± 0 | 1500.0 ± 0% | \+62.87% (p=0.000 n=10) |
| GetHeader-8 | 1.842k ± 0% | 2.407k ± 0% | \+30.67% (p=0.000 n=10) |
| GetPayload-8 | 4.239k ± 0% | 2.407k ± 0% | \-51.62% (p=0.000 n=10) |
| geomean | 1.930k | 1.949k | \+0.98% |