swift-server / swift-server/async-http-client

Support opt-in gzip/deflate compression for request bodies

Open
#917 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Swift
Stars
1.1k
Forks
156
PR merge metrics
No merged PRs in 30d

Description

Summary

Please add a supported, opt-in API for gzip/deflate compression of outgoing request bodies.

I checked the current main branch at commit
f95c908967e98c68c5ce3fd61a7974e7e869e303.

Motivation

AsyncHTTPClient supports automatic gzip/deflate decompression of response
bodies through HTTPClient.Configuration.decompression, but I could not find
an equivalent API for compressing outgoing request bodies in
HTTPClientRequest, HTTPClient.Request, or HTTPClient.Configuration.

My concrete use case is uploading large, highly compressible generated
artifacts from AWS Lambda to a Cloudflare Worker, which then stores them in R2.

The outbound network boundary is metered by transferred bytes, and the data
transfer cost can exceed the Lambda compute cost. Opt-in gzip compression can
materially reduce the number of bytes sent.

This is also useful for service-to-service uploads of JSON, HTML, logs, source
snapshots, and other highly compressible payloads.

Current workarounds

As far as I can tell, callers currently need to choose one of these approaches:

  1. Pre-compress the body and set Content-Encoding: gzip manually.

    This requires callers to provide a compression implementation and to handle
    streaming, backpressure, compressed body length, and replayability
    themselves.

  2. Add NIOHTTPRequestCompressor through AsyncHTTPClient's debug pipeline
    initializers.

    This requires protocol-specific pipeline manipulation and does not provide a
    supported per-request abstraction. A recent implementation in
    request-dl-nio uses http1_1ConnectionDebugInitializer, so that workaround
    is limited to HTTP/1.1.

AsyncHTTPClient already depends on swift-nio-extras's NIOHTTPCompression
product, and NIOHTTPRequestCompressor already implements streaming
gzip/deflate compression while updating Content-Encoding and HTTP/1 framing.

Possible API

The exact API shape is open for discussion, but a per-request option seems
safer than enabling compression for every request:

public enum RequestBodyCompression: Sendable {
    case disabled
    case gzip
    case deflate
}

var request = HTTPClientRequest(url: uploadURL)
request.method = .PUT
request.body = .stream(body, length: .unknown)
request.requestBodyCompression = .gzip

let response = try await client.execute(request, timeout: .seconds(30))

A client-wide default with a per-request override might also be useful, but
compression should remain disabled by default because many payloads are already
compressed or too small to benefit.

Expected behavior

  • Support both in-memory and streaming request bodies.

  • Preserve AsyncHTTPClient's existing backpressure behavior.

  • Set Content-Encoding automatically.

  • Set framing based on the compressed representation:

    • Content-Length when the compressed length is known.
    • Protocol-appropriate streaming framing when it is unknown.
  • Work with both HTTP/1.1 and HTTP/2.

  • Define or reject conflicts with an existing Content-Encoding header rather
    than silently double-compressing.

  • Preserve the existing redirect, retry, and body replayability semantics.

  • Do not add Content-Encoding when there is no request body.

Because NIOHTTPRequestCompressor performs compression on the event-loop
thread, this should remain opt-in and should be documented primarily for large,
highly compressible request bodies.

Related work

Would the maintainers prefer a dedicated request-body compression API, or
should this use case be handled by a general request-body transformation or
middleware API related to #393?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading HTTPClientRequest, HTTPClient.Request, and HTTPClient.Configuration, then inspect swift-nio-extras' NIOHTTPCompression/HTTPRequestCompressor.swift and the middleware proposal in issue #393. Define the supported API and its interaction with streaming, backpressure, framing, headers, HTTP/1.1, HTTP/2, retries, redirects, and replayability; done means these behaviors are specified and covered for in-memory and streaming bodies.

Written by the indexing model from the issue text.

Assessment

Tech stack
swift
Domain
api
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.