[Feature proposal] Load balancer improvements and customisation

Rationale

The current load balancer is hard-coded to be "pseudo-ping"-based. This solution has several limitations:

1. Ping does not accurately reflect server "quality". For example, RTT is heavily dependent on geographical location. A server might be physically closer to the client and therefore has better ping, but actually offers very limited throughput. (This is actually my case: my VPS in San Jose has significantly higher bandwidth than my VPS in Taiwan, but the current load balancer prefers the later 😓.)
2. The current implementation is not a "true" ping, as it includes the RTT between the server and known endpoints (TCP: Firefox portal detection URL; UDP: Google DNS). This could potentially make the RTT less accurate, although admittedly it's an unlikely scenario.
3. Total bandwidth usage is not taken into account. One common use of load balancing is to distribute the data evenly (more or less evenly) across multiple endpoints (servers), or to set bandwidth caps for each endpoint.
4. General lack of configurability and extensibility.

What I want to accomplish in this proposal

Load balancing is a very complex subject, so there's a literal ton of things that we need to go through and decide on.

Therefore it is not my goal in this proposal (at least initially) to discuss any implementation details. Instead I would like to ask the community to brainstorm ideas regarding:

1. What sub-features of load balancing do we need to support?
2. How do we structure those sub-features in a way that is user-customisable (configurability) and somewhat future proof (extensibility)?

Of course we should still keep implementation in mind during the brainstorm. I.e. don't propose schemes that takes an entire Microsoft to develop.

General structure of the load balancing system I am envisioning

I guess the best way to illustrate this is to write some pseudo-Rust code.

This is very much my own early-stage idea, so feel free to disagree.

// Each variant is one factor that the ranking system can consider (i.e. metric)
// Each variant of Metric contains its score (and potentially other data)
enum Metric {
  ClientToServerPing(u32),
  ServerToEndpointPing(u32),
  DataRate(u64),
  DataCapReached(bool),
  ...
}

// This function returns the best server choice for this moment
//
// The load balancer calls this function every 5 seconds (or whatever interval the user specifies),
// and uses this ServerProfile for all requests until the next time this function is called
//
fn get_best_server(servers: &[ServerProfile], opts: LoadBalancer::UserOptions) -> &ServerProfile {

  let servers_metrics: Vec<Vec<Metric>> = servers.iter()
    .map(|sp: &ServerProfile| {
      // For each server profile, check its scores for all metrics defined by user
      let metrics: Vec<Metric> = opts.get_chosen_metric_suite().check_server(sp);
      return metrics;
    })
    .collect();

  let servers_final_score: Vec<SynthesisedScore> = servers_metrics.iter()
    .map(|v_m: &Vec<Metric>| {
      // For each server profile's metrics, generate an overall score (synthesise)
      let score: SynthesisedScore = opts.get_chosen_synthesiser().gen_score(v_m);
      return score;
    })
    .collect();

  let best_idx = servers_final_score.into_iter().enumerate().max_by_key(|(i, &&score)| score).unwrap().0;
  return &servers[best_idx];
}

Potential ranking metrics we can consider supporting

1. Ping between client and each server
2. Ping between each server and known endpoints
3. Total data usage for each server
4. Current data rate (inbound & outbound) for each server
5. User-assigned priority for each server

Feel free to suggest other potential metrics I missed.

How to build the score synthesiser?

Naive single-metric ranker

Quite self-explanatory. Only lets the user choose one metric, and just uses that metric's score to rank.

This could also be used as a baby-step towards a more comprehensive system.

A more comprehensive, user-configurable ranker

Honestly, I haven't made up my mind yet here. There are so many possibilities.

• A weighted-average system? (e.g. ping & data rate)
• With a "disqualification layer" placed in front? (e.g. data cap reached)
• What about hysteresis? (to deal with instability in network conditions)

Maybe this last step can wait. Suggestions are more than welcomed, of course.

Start by reviewing crates/shadowsocks-service/src/local/loadbalancing/ping_balancer.rs, especially the TCP and UDP probing sections linked in the proposal, and compare them with the suggested Metric and get_best_server structures. The issue does not define an implementation target or acceptance criteria, so the first step is to narrow the desired metrics, configuration model, and scoring behavior before coding.