Orchestrator selection mechanism

In the Livepeer marketplace, broadcasters send video segments to orchestrators who return transcoded results based on the transcoding options requested by a broadcaster. Broadcasters need a mechanism for selecting orchestrators. This mechanism is important because it:

- Affects how quickly a broadcaster can connect to an orchestrator that is able to provide high quality service. Even though a broadcaster can always switch orchestrators in the event of the failure of an existing orchestrator or distribute video segments to multiple orchestrators for increased resiliency, a broadcaster would benefit from selecting high quality orchestrators with a high probability
- Affects how transcoding fees are routed to orchestrators which can impact the economics of staking

The first point is critical for providing real value to users of the Livepeer network. We highlight quality of service because there is a difference between correctly doing the requested transcoding and doing the requested transcoding in a timely manner that reduces the latency for the broadcaster’s stream. So, while a verification protocol can be used to check transcoding work done by orchestrators, broadcasters will potentially still need to evaluate orchestrators a long other dimensions such as latency and reputation in addition to the economic security provided by an orchestrator in the form of stake that can be slashed upon failed verification.

The second point is critical for understanding the long term economics of staking and its role in securing and enabling an open marketplace for video transcoding. We acknowledge that an optimal orchestrator selection mechanism will accept a variety of inputs that capture potentially useful informational signals that can be used to evaluate orchestrators and output a list of candidate orchestrators to work with (the base case is a list of size 1, but the list can be of size N if a broadcaster desires increased resiliency by distributing video segments to multiple orchestrators). As mentioned previously, some of these inputs include economic security, latency and reputation. So, key questions to answer include:

- How should each of these inputs weighed?
- How is each input used to filter out orchestrators?
- Economic security can be measured via on-chain stake and latency can be measured in milliseconds via network request based speed tests, but should reputation be measured? If so, how should it be measured?

The problem we are interested in solving is constructing a orchestrator selection mechanism based on economic security, latency and reputation (and possibly other factors if they are deemed important) that is usable in the short term, but also amenable to updates in the long term that involve more sophisticated evaluation techniques (i.e. a more sophisticated method for calculating reputation scores).

Find below a candidate construction:

We will refer to broadcasters as Bs and orchestrators as Os for convenience.

- B periodically refreshes its cached list of Os along with their price menus (include pricing information for various transcoding options) and latency measurements
- B defines a max price for a set of transcoding options when it creates a stream
- B can derive each O's price for the requested transcoding options using each O's price menu
- B selects a subset of Os based on their derived prices and its max price
- From this subset, B selects the top N Os with the lowest recorded latency measurements
- From this subset, B selects the top M Os with a cumulative rating score that is computed using O’s stake and reputation score
- B randomly samples X Os from the entire list of Os (by default X = 1 and can be configured to be higher) and adds them to this subset. We do this to give new Os and lower rating Os the opportunity to be selected. The end result is the selection set
- B queries all the Os in the selection set and picks the first O that responds (or first few responses if redundancy/increased resiliency is desired)

**Open Problems/Questions**

- How is the reputation score calculated?
- How can stake and reputation score be combined to compute a cumulative rating score?
- How are stake and reputation score weighted in computing a cumulative rating score?
- Is it reasonable to think that when comparing Os, less stake will require more reputation while more stake will require less reputation?
- Should Bs define minimum stake (minimum economic security) and/or minimum reputation scores?

**Additional Thoughts**

Latency measurements can help with guarantees around quality of service without requiring knowledge of specific geographic regions for orchestrators. But, perhaps geographic region can be useful in certain circumstances where a broadcaster wants video to only be sent to a certain geographic area (perhaps due to concerns around rules of a jurisdiction?). Geo IP look ups on resolved orchestrator hostnames may help here.

No implementation files, tests, or entry points are identified. Start by reviewing the candidate construction and its open questions around reputation, stake weighting, latency, and selection; done means an agreed, implementable orchestrator-selection mechanism with defined inputs and scoring rules.

Written by the indexing model from the issue text.