apache / apache/pekko-http

HTTP/2 client: allow `managedPersistentHttp2` to balance over multiple connections

Open
#1,248 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Scala
Stars
196
Forks
55
Avg merge
4d 52m
Merged PRs (30d)
74

Description

## Problem

There is currently no way to ask the HTTP/2 client for more than one live connection to a
single authority, which makes client-side load balancing impossible to build on top of it.

pekko-grpc hits this directly. Its `pekko-http` client backend resolves a service name to a
set of endpoints via Pekko Discovery and is supposed to spread requests over them
(`round_robin`, as the `netty` backend does via grpc-java). What it can do today is in
[`PekkoHttpClientUtils`](https://github.com/apache/pekko-grpc/blob/main/runtime/src/main/scala/org/apache/pekko/grpc/internal/PekkoHttpClientUtils.scala#L71-L111):

- a `ClientTransport.withCustomResolver` that round-robins over the discovered addresses, and
- `builder.managedPersistentHttp2()` feeding a single `Source.queue`.

Because `managedPersistentHttp2` maintains exactly one connection, the resolver is only
consulted when that connection is (re)established. So every request from a given client goes
to whichever endpoint was picked at connect time; the counter buys failover across reconnects,
not balancing. pekko-grpc's `LoadBalancingIntegrationSpec` is
[commented out](https://github.com/apache/pekko-grpc/blob/main/interop-tests/src/test/scala/org/apache/pekko/grpc/scaladsl/LoadBalancingIntegrationSpec.scala#L41-L46)
for the `pekko-http` backend for this reason, and `GrpcClientSettings.withLoadBalancingPolicy`
is silently ignored there.

This is the pekko side of akka/akka-grpc#1197 ("Client-side loadbalancing with the Akka HTTP
backend"), which is still open upstream with the API question unanswered.

## Why the existing issue doesn't cover it

#484 asks for an HTTP/2 pool behind `singleRequest`, described there as "keeps a single
managed persistent HTTP/2 connection **per host** and automatically dispatches requests based
on the host to that connection". That is a different axis: it gives one connection per
distinct authority. Here all the endpoints share one authority (`greeter`, or whatever
`overrideAuthority` says) and differ only in the resolved socket address, so a per-host pool
would still collapse them onto one connection.

## Proposed fix

Add a multi-connection variant of the managed persistent flow to `OutgoingConnectionBuilder`,
keeping the existing `Flow[HttpRequest, HttpResponse, NotUsed]` shape so it is a drop-in
replacement:

```scala
@ApiMayChange
def managedPersistentHttp2(connections: Int): Flow[HttpRequest, HttpResponse, NotUsed]

@ApiMayChange
def managedPersistentHttp2WithPriorKnowledge(connections: Int): Flow[HttpRequest, HttpResponse, NotUsed]
```

Implementation is mostly wiring in `OutgoingConnectionBuilderImpl`: materialize `connections`
independent `PersistentConnection.managedConnection(http2(), ...)` sub-flows, fan requests out
over them and merge the responses back. Correlating responses already works — HTTP/2 requests
must carry a `RequestResponseAssociation`, and `PersistentConnection` already relies on that,
so an unordered merge is safe.

The key property for the load-balancing use case is that each sub-flow establishes its own
connection through the configured `ClientTransport`, so a caller-supplied custom resolver is
invoked once per connection and can hand out a different endpoint each time. pekko-grpc then
gets round-robin over discovered endpoints without any new discovery API in pekko-http, and
can enable the spec that is currently commented out.

## Design questions

- **Fan-out strategy.** A plain `Balance` is the cheap option but routes to whichever sub-flow
pulls first. A `PersistentConnection` that is unconnected or sitting in its reconnect embargo
will still accept a request and buffer it, so `Balance` can steer traffic *towards* a broken
endpoint. Something that prefers connected sub-flows, or routes by fewest outstanding
requests, would behave much better and is closer to what gRPC's `round_robin` does.
- **Should the connection count be dynamic?** Fixed `connections: Int` is the small version.
Resizing as the discovered endpoint set changes is the fuller answer, but needs pekko-http
to know about endpoint sets — which is the "how much of this should be a new API on the HTTP
side" question from akka/akka-grpc#1197. Suggest starting fixed.
- **Config.** Optionally a `pekko.http.client.http2.connections` default alongside the existing
`max-persistent-attempts` / `base-connection-backoff` keys, though the builder argument may be
enough given this is a per-client decision.
- **javadsl parity** in `pekko.http.javadsl.OutgoingConnectionBuilder` and its adapter.
- `connections = 1` should stay behaviourally identical to today's overload.

Both existing methods are already `@ApiMayChange`, so adding overloads here is cheap.

## Alternatives considered

pekko-grpc could manage N `managedPersistentHttp2` flows itself and round-robin `singleRequest`
across them, entirely inside `PekkoHttpClientUtils` and with no pekko-http change. That works,
but it reimplements connection health and eviction logic in a downstream project, and any other
user wanting HTTP/2 client-side load balancing would have to do the same.

Contributor guide

Open the contributing guide

Research direction

Start by reading OutgoingConnectionBuilder and OutgoingConnectionBuilderImpl, then trace PersistentConnection.managedConnection and the existing HTTP/2 methods. Define the multi-connection fan-out and response merge, preserve the connections = 1 behavior, and cover the matching javadsl builder adapter; completion should allow independent connections to use separate resolved endpoints.

Written by the indexing model from the issue text.

Assessment

Tech stack
scala
Domain
api, backend, networking
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.