HTTP/2 client: allow `managedPersistentHttp2` to balance over multiple connections
- Dominant language
- Scala
- Stars
- 196
- Forks
- 55
- Avg merge
- 4d 52m
- Merged PRs (30d)
- 74
Description
## Problem
There is currently no way to ask the HTTP/2 client for more than one live connection to a
single authority, which makes client-side load balancing impossible to build on top of it.
pekko-grpc hits this directly. Its `pekko-http` client backend resolves a service name to a
set of endpoints via Pekko Discovery and is supposed to spread requests over them
(`round_robin`, as the `netty` backend does via grpc-java). What it can do today is in
[`PekkoHttpClientUtils`](https://github.com/apache/pekko-grpc/blob/main/runtime/src/main/scala/org/apache/pekko/grpc/internal/PekkoHttpClientUtils.scala#L71-L111):
- a `ClientTransport.withCustomResolver` that round-robins over the discovered addresses, and
- `builder.managedPersistentHttp2()` feeding a single `Source.queue`.
Because `managedPersistentHttp2` maintains exactly one connection, the resolver is only
consulted when that connection is (re)established. So every request from a given client goes
to whichever endpoint was picked at connect time; the counter buys failover across reconnects,
not balancing. pekko-grpc's `LoadBalancingIntegrationSpec` is
[commented out](https://github.com/apache/pekko-grpc/blob/main/interop-tests/src/test/scala/org/apache/pekko/grpc/scaladsl/LoadBalancingIntegrationSpec.scala#L41-L46)
for the `pekko-http` backend for this reason, and `GrpcClientSettings.withLoadBalancingPolicy`
is silently ignored there.
This is the pekko side of akka/akka-grpc#1197 ("Client-side loadbalancing with the Akka HTTP
backend"), which is still open upstream with the API question unanswered.
## Why the existing issue doesn't cover it
#484 asks for an HTTP/2 pool behind `singleRequest`, described there as "keeps a single
managed persistent HTTP/2 connection **per host** and automatically dispatches requests based
on the host to that connection". That is a different axis: it gives one connection per
distinct authority. Here all the endpoints share one authority (`greeter`, or whatever
`overrideAuthority` says) and differ only in the resolved socket address, so a per-host pool
would still collapse them onto one connection.
## Proposed fix
Add a multi-connection variant of the managed persistent flow to `OutgoingConnectionBuilder`,
keeping the existing `Flow[HttpRequest, HttpResponse, NotUsed]` shape so it is a drop-in
replacement:
```scala
@ApiMayChange
def managedPersistentHttp2(connections: Int): Flow[HttpRequest, HttpResponse, NotUsed]
@ApiMayChange
def managedPersistentHttp2WithPriorKnowledge(connections: Int): Flow[HttpRequest, HttpResponse, NotUsed]
```
Implementation is mostly wiring in `OutgoingConnectionBuilderImpl`: materialize `connections`
independent `PersistentConnection.managedConnection(http2(), ...)` sub-flows, fan requests out
over them and merge the responses back. Correlating responses already works — HTTP/2 requests
must carry a `RequestResponseAssociation`, and `PersistentConnection` already relies on that,
so an unordered merge is safe.
The key property for the load-balancing use case is that each sub-flow establishes its own
connection through the configured `ClientTransport`, so a caller-supplied custom resolver is
invoked once per connection and can hand out a different endpoint each time. pekko-grpc then
gets round-robin over discovered endpoints without any new discovery API in pekko-http, and
can enable the spec that is currently commented out.
## Design questions
- **Fan-out strategy.** A plain `Balance` is the cheap option but routes to whichever sub-flow
pulls first. A `PersistentConnection` that is unconnected or sitting in its reconnect embargo
will still accept a request and buffer it, so `Balance` can steer traffic *towards* a broken
endpoint. Something that prefers connected sub-flows, or routes by fewest outstanding
requests, would behave much better and is closer to what gRPC's `round_robin` does.
- **Should the connection count be dynamic?** Fixed `connections: Int` is the small version.
Resizing as the discovered endpoint set changes is the fuller answer, but needs pekko-http
to know about endpoint sets — which is the "how much of this should be a new API on the HTTP
side" question from akka/akka-grpc#1197. Suggest starting fixed.
- **Config.** Optionally a `pekko.http.client.http2.connections` default alongside the existing
`max-persistent-attempts` / `base-connection-backoff` keys, though the builder argument may be
enough given this is a per-client decision.
- **javadsl parity** in `pekko.http.javadsl.OutgoingConnectionBuilder` and its adapter.
- `connections = 1` should stay behaviourally identical to today's overload.
Both existing methods are already `@ApiMayChange`, so adding overloads here is cheap.
## Alternatives considered
pekko-grpc could manage N `managedPersistentHttp2` flows itself and round-robin `singleRequest`
across them, entirely inside `PekkoHttpClientUtils` and with no pekko-http change. That works,
but it reimplements connection health and eviction logic in a downstream project, and any other
user wanting HTTP/2 client-side load balancing would have to do the same.
Contributor guide
Research direction
Start by reading OutgoingConnectionBuilder and OutgoingConnectionBuilderImpl, then trace PersistentConnection.managedConnection and the existing HTTP/2 methods. Define the multi-connection fan-out and response merge, preserve the connections = 1 behavior, and cover the matching javadsl builder adapter; completion should allow independent connections to use separate resolved endpoints.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- scala
- Domain
- api, backend, networking
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100