googleapis / googleapis/google-cloud-swift

Consider a round-robin decorator for the storage client

Open
#651 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Swift
Stars
26
Forks
10
Avg merge
12h 56m
Merged PRs (30d)
211

Description

Gemini says that AsyncHTTPClient opens at most 1 or 2 connections for HTTP/2. That is not enough for a high throughput client like Storage. We need to round-robin across multiple AsyncHTTPClients, or use something like the p2c algorithm to load balance based on load.

----
My prompt:

Find some explicit documentation link, or code comments explaining how many connections are opened by AsyncHTTPClient. Provide the links and relevant text. Prevent hallucinations: verify the text by downloading from the link and comparing the text you are highlighting.

----

### 1. HTTP/1.1 Connection Limit: Default 8 Connections per Host

The default limit for concurrent HTTP/1.1 connections per host is **8**, configured via `HTTPClient.Configuration.ConnectionPool.concurrentHTTP1ConnectionsPerHostSoftLimit`.

* **Source File & Link**: [`Sources/AsyncHTTPClient/HTTPClient.swift` (lines 1394–1424)](https://github.com/swift-server/async-http-client/blob/1.36.0/Sources/AsyncHTTPClient/HTTPClient.swift#L1394-L1424)
* **Verbatim Code & Docstrings**:

```swift
/// Connection pool configuration.
public struct ConnectionPool: Hashable, Sendable {
/// Specifies amount of time connections are kept idle in the pool. After this time has passed without a new
/// request the connections are closed.
public var idleTimeout: TimeAmount = .seconds(60)

/// The maximum number of connections that are kept alive in the connection pool per host. If requests with
/// an explicit eventLoopRequirement are sent, this number might be exceeded due to overflow connections.
public var concurrentHTTP1ConnectionsPerHostSoftLimit: Int = 8

/// If true, ``HTTPClient`` will try to create new connections on connection failure with an exponential backoff.
/// Requests will only fail after the ``HTTPClient/Configuration/Timeout-swift.struct/connect`` timeout exceeded.
/// If false, all requests that have no assigned connection will fail immediately after a connection could not be established.
/// Defaults to `true`.
/// - warning: We highly recommend leaving this on.
/// It is very common that connections establishment is flaky at scale.
/// ``HTTPClient`` will automatically mitigate these kind of issues if this flag is turned on.
public var retryConnectionEstablishment: Bool = true

/// The number of pre-warmed HTTP/1 connections to maintain.
///
/// When set to a number greater than zero, any HTTP/1 connection pool created will attempt to maintain
/// at least this number of "extra" idle connections, above the connections currently in use, up to the
/// limit of ``concurrentHTTP1ConnectionsPerHostSoftLimit``.
///
/// These connections will not be made while the pool is idle: only once the first connection is made
/// to a host will the others be opened. In addition, to manage the connection creation rate and
/// avoid flooding servers, prewarmed connection creation will be done one-at-a-time.
public var preWarmedHTTP1ConnectionCount: Int = 0
```

#### Enforcement in the HTTP/1.1 State Machine
In [`Sources/AsyncHTTPClient/ConnectionPool/State Machine/HTTPConnectionPool+HTTP1Connections.swift` (lines 307–309)](https://github.com/swift-server/async-http-client/blob/1.36.0/Sources/AsyncHTTPClient/ConnectionPool/State%20Machine/HTTPConnectionPool%2BHTTP1Connections.swift#L307-L309), the state machine enforces this ceiling:

```swift
var canGrow: Bool {
self.overflowIndex < self.maximumConcurrentConnections
}
```
Where `maximumConcurrentConnections` is initialized directly from `concurrentHTTP1ConnectionsPerHostSoftLimit`.

---

### 2. HTTP/2 Connection Behavior: Multiplexing over Existing Connections

For HTTP/2 (which is negotiated by default over TLS when connecting to endpoints like `storage.googleapis.com`), `AsyncHTTPClient` **multiplexes multiple concurrent requests (streams) over an existing connection** instead of opening a new TCP socket for each request.

* **Source File & Link**: [`Sources/AsyncHTTPClient/ConnectionPool/State Machine/HTTPConnectionPool+HTTP2StateMachine.swift` (lines 163–180)](https://github.com/swift-server/async-http-client/blob/1.36.0/Sources/AsyncHTTPClient/ConnectionPool/State%20Machine/HTTPConnectionPool%2BHTTP2StateMachine.swift#L163-L180)
* **Verbatim Code & Comments**:

```swift
/// 2. No available stream so we definitely need to wait until we have one
self.requests.push(request)

if self.connections.hasConnectionThatCanOrWillBeAbleToExecuteRequests(for: eventLoop) {
/// 3. we already have a connection, we just need to wait until until it becomes available
return .init(
request: .scheduleRequestTimeout(for: request, on: eventLoop),
connection: .none
)
} else {
/// 4. we do *not* have a connection, need to create a new one and wait until it is connected.
let connectionId = self.connections.createNewConnection(on: eventLoop)
return .init(
request: .scheduleRequestTimeout(for: request, on: eventLoop),
connection: .createConnection(connectionId, on: eventLoop)
)
}
```

* **Behavior**:
- If a connection already exists on that EventLoop, `AsyncHTTPClient` does **not** create a new connection (`connection: .none`). It waits for a stream to become available on the existing HTTP/2 connection.
- Sockets only scale up to the number of distinct `EventLoop`s processing requests, typically **1 connection per host** (or at most one per active EventLoop in the thread pool).

---

### 3. Verification in Unit Tests

`AsyncHTTPClient` has a dedicated test suite verifying that `concurrentHTTP1ConnectionsPerHostSoftLimit` is strictly respected even under high concurrency.

* **Source File & Link**: [`Tests/AsyncHTTPClientTests/ConnectionPoolSizeConfigValueIsRespectedTests.swift` (lines 34–76)](https://github.com/swift-server/async-http-client/blob/1.36.0/Tests/AsyncHTTPClientTests/ConnectionPoolSizeConfigValueIsRespectedTests.swift#L34-L76)
* **Verbatim Test Code**:

```swift
final class ConnectionPoolSizeConfigValueIsRespectedTests: XCTestCaseHTTPClientTestsBaseClass {
func testConnectionPoolSizeConfigValueIsRespected() {
let numberOfRequestsPerThread = 1000
let numberOfParallelWorkers = 16
let poolSize = 12
...
let configuration = HTTPClient.Configuration(
connectionPool: .init(
idleTimeout: .seconds(30),
concurrentHTTP1ConnectionsPerHostSoftLimit: poolSize
)
)
let client = HTTPClient(eventLoopGroupProvider: .shared(group), configuration: configuration)
...
// 16 workers execute 1,000 requests each (16,000 total requests)
...
XCTAssertEqual(httpBin.createdConnections, poolSize)
}
}
```

This verifies that even with 16 parallel threads making 16,000 requests, the total number of TCP connections created to the target host never exceeds `poolSize` (default: 8).

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the cited AsyncHTTPClient.swift connection-pool configuration, HTTPConnectionPool+HTTP1Connections.swift, HTTPConnectionPool+HTTP2StateMachine.swift, and ConnectionPoolSizeConfigValueIsRespectedTests.swift. Verify the documented HTTP/1.1 and HTTP/2 behavior, then determine whether the Storage client needs round-robin or p2c balancing and what measurable connection behavior would constitute a complete decision.

Written by the indexing model from the issue text.

Assessment

Tech stack
google-cloud, swift
Domain
cloud, performance
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.