googleapis / googleapis/google-cloud-swift
storage: reduce resumable upload HTTP transfers and eliminate inter-chunk latency
- Dominant language
- Swift
- Stars
- 26
- Forks
- 10
- Avg merge
- 12h 56m
- Merged PRs (30d)
- 211
Description
# storage: reduce resumable upload HTTP transfers and eliminate inter-chunk latency
## Problem Description
In `pkgs/swift-google-cloud-storage/Sources/GoogleCloudStorage/StorageClient+Upload.swift`, resumable uploads for seekable sources (such as local files and in-memory byte buffers) are artificially split into sequential fixed-size HTTP PUT requests (defaulting to 8 MiB, or 32 MiB in custom configurations).
Each intermediate chunk transfer:
1. Streams the chunk body and terminates the HTTP request.
2. Waits for Google Cloud Storage to commit the chunk and return `HTTP/1.1 308 Resume Incomplete`.
3. Pauses transmission while the client parses headers, drains the response, and constructs a new `_HTTPClientRequest`.
4. Sends new HTTP request headers and awaits socket write buffer availability.
During every chunk boundary transition, **network transmission is completely idle for 20–50 ms** (composed of round-trip network latency plus server-side commit and turnaround time). In addition to the idle dead time, terminating and recreating HTTP streams disrupts TCP congestion window (`cwnd`) pacing and HTTP/2 stream flow control.
---
## Quantitative Impact Analysis (128 MiB and 256 MiB Uploads)
Assuming a baseline network link capable of **100 MiB/s** (800 Mbps) and a typical in-region server commit + RTT pause of **30 ms** per chunk boundary (with an initial session creation POST of 25 ms):
### 1. 128 MiB Upload
| Configuration | HTTP Transfers | Boundary Pauses ($30\text{ ms}$ each) | Payload Tx Time | Total Time | Effective Throughput | Throughput Penalty |
| :--- | :--- | :--- | :--- | :--- | :--- | :--- |
| **8 MiB chunks** | **17** (1 POST + 16 PUTs) | $15 \times 30\text{ ms} = \mathbf{450\text{ ms}}$ | 1,280 ms | 1,755 ms | **72.9 MiB/s** | **-27.1%** |
| **32 MiB chunks** | **5** (1 POST + 4 PUTs) | $3 \times 30\text{ ms} = \mathbf{90\text{ ms}}$ | 1,280 ms | 1,395 ms | **91.8 MiB/s** | **-8.2%** |
| **Single Streaming PUT** | **2** (1 POST + 1 PUT) | $\mathbf{0\text{ ms}}$ | 1,280 ms | 1,305 ms | **98.1 MiB/s** | **~0%** |
*(If server turnaround/RTT is 50 ms, 8 MiB chunks drop to **62.3 MiB/s**, and 32 MiB chunks drop to **88.0 MiB/s**).*
### 2. 256 MiB Upload
| Configuration | HTTP Transfers | Boundary Pauses ($30\text{ ms}$ each) | Payload Tx Time | Total Time | Effective Throughput | Throughput Penalty |
| :--- | :--- | :--- | :--- | :--- | :--- | :--- |
| **8 MiB chunks** | **33** (1 POST + 32 PUTs) | $31 \times 30\text{ ms} = \mathbf{930\text{ ms}}$ | 2,560 ms | 3,515 ms | **72.8 MiB/s** | **-27.2%** |
| **32 MiB chunks** | **9** (1 POST + 8 PUTs) | $7 \times 30\text{ ms} = \mathbf{210\text{ ms}}$ | 2,560 ms | 2,795 ms | **91.6 MiB/s** | **-8.4%** |
| **Single Streaming PUT** | **2** (1 POST + 1 PUT) | $\mathbf{0\text{ ms}}$ | 2,560 ms | 2,585 ms | **99.0 MiB/s** | **~0%** |
*(If server turnaround/RTT is 50 ms, 8 MiB chunks drop to **61.9 MiB/s**, and 32 MiB chunks drop to **87.2 MiB/s**).*
### Summary of Impact
- With **8 MiB chunks**, over **27% of available bandwidth is lost** purely to round-trip stops and starts, capping uploads at ~72 MiB/s regardless of file size.
- Increasing chunk size to **32 MiB** recovers much of this loss (~91 MiB/s), but still incurs an **8% penalty** and introduces bursty traffic patterns.
- **Single Streaming PUT** eliminates all intermediate pauses, allowing the TCP pipe to remain fully saturated at **> 98 MiB/s**.
---
## Proposed Solution
1. **Default to Single-Request Streaming for Seekable Sources**:
In `StorageClient+Upload.swift`, when uploading a `SeekableUploadSource` with known `totalSize`, and `options.chunkSize == nil` (or unbuffered mode):
- Send the entire object in a single PUT request:
`Content-Range: bytes 0-(totalSize - 1)/totalSize`
`Content-Length: totalSize`
- Stream the data in 2 MiB chunks via `request.setBody(stream:length:)`, computing CRC32C on the fly.
- If a network interruption occurs, query `queryUploadStatus`, seek the source to the server's committed byte offset, and stream the remaining bytes in another PUT request.
2. **Preserve Chunked Mode When Explicitly Requested**:
If an application explicitly specifies `options.chunkSize = N` (e.g. `options.chunkSize = 32 * 1024 * 1024`), preserve the chunked behavior for applications that rely on explicit chunk boundaries.
---
## Changes to `StorageW1R3` to Measure Effect
To benchmark and validate this optimization, update `Tests/StorageW1R3`:
1. **Add Command-Line Flags in `StorageW1R3.swift`**:
```swift
@Option(
name: .customLong("resumable-mode"),
help: "Resumable upload mode: streaming (single PUT), chunked (fixed/random chunks), or random."
)
var resumableMode: ResumableModeOption = .streaming
```
2. **Update `StorageOperations.swift`**:
```swift
if isResumable {
switch resumableMode {
case .streaming:
$0.chunkSize = nil // Unbuffered single streaming PUT
case .chunked:
$0.chunkSize = configuredChunkSize // e.g. 32 MiB or 8 MiB
}
$0.resumableUploadThreshold = buffer.readableBytes
}
```
3. **Record Mode in `Sample.swift`**:
Add `ResumableMode` (or include `mode=streaming` / `mode=chunked` in the `Details` column) so BigQuery analysis can directly compare throughput and latency between single-stream and chunked uploads.
---
## Acceptance Criteria
- Resumable uploads with known total size execute in exactly **2 HTTP transfers** (1 POST session creation + 1 streaming PUT) when `chunkSize` is not set.
- 128 MiB and 256 MiB uploads achieve **> 90 MiB/s** throughput under single-stream mode.
- Explicit `chunkSize` settings continue to function as chunked uploads.
- `StorageW1R3` can be invoked with `--resumable-mode streaming` vs `--resumable-mode chunked` to measure and compare throughput.
Contributor guide
Research direction
Start in pkgs/swift-google-cloud-storage/Sources/GoogleCloudStorage/StorageClient+Upload.swift and trace resumable handling for SeekableUploadSource, including queryUploadStatus and chunkSize behavior. Then inspect Tests/StorageW1R3/StorageW1R3.swift, StorageOperations.swift, and Sample.swift for benchmark mode handling. Done means unconfigured uploads use one streaming PUT after session creation, explicit chunk sizes remain chunked, and streaming versus chunked measurements are recorded.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- swift
- Domain
- api, cloud
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100