cloudflare / cloudflare/pingora

Expose configurable HTTP/1 request-header parser admission limits

Open
#993 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
27.4k
Forks
1.7k
Avg merge
6h 22m
Merged PRs (30d)
3

Description

## What is the problem your feature solves, or the need it fulfills?

At current `main` (`09696b51bc59315353d96686355861604d0bb48c`), downstream HTTP/1 request-header admission is fixed inside `pingora-core`:

- `pingora-core/src/protocols/http/v1/common.rs` defines `MAX_HEADERS = 256`, `INIT_HEADER_BUF_SIZE = 4096`, and `MAX_HEADER_SIZE = 1_048_575`.
- `HttpSession::read_request()` in `pingora-core/src/protocols/http/v1/server.rs` grows its parser buffer against that fixed ceiling. The source notes that the size check happens before the next read, so a completed large socket read can already exceed the nominal threshold before rejection.
- `HttpServerOptions` currently exposes transport options such as h2c, keepalive request limits, and H2 idle timeout, but no HTTP/1 request-header byte/count admission setting.

For an edge proxy that needs an operator-controlled header budget materially below the built-in ceiling, rejecting in a `ProxyHttp::request_filter()` callback is too late to serve as a parser/resource admission control: the request header has already been buffered and parsed before that callback.

HTTP/2 already has a different control surface: `default_h2_options()` applies a decoded header-list limit and `H2Options::max_header_list_size` can customize it. I am not proposing that HTTP/1 wire/parser bytes and HTTP/2 decoded header-list size be treated as identical accounting.

## Describe the solution you'd like

Please expose a supported HTTP/1 server/composition option that can lower request-header resource admission before or during parser buffering, without requiring downstream users to copy or patch Pingora parser source.

A useful API would preserve today's behavior when unset and allow an adopter to configure, at minimum:

- a positive maximum HTTP/1 request-header byte budget; and
- preferably a positive maximum HTTP/1 header-field count, bounded by the existing safe supplier maximum.

The limit should be applied by the HTTP/1 parser path itself, including pipelined-prefix handling, rather than only after `RequestHeader` construction. Oversized traffic may fail at the transport/parser boundary; a specific application status such as 431 is less important than enforcing the resource bound before application callbacks.

Expected compatibility/acceptance properties:

1. `None`/unset preserves the current constants and behavior.
2. Near-limit HTTP/1 requests succeed; over-limit one-large-field and many-small-fields requests fail before application callback processing.
3. The configured byte/count limits also apply when the request begins from a pipelined prefix on a reused connection.
4. Zero/impossible values fail configuration rather than silently disabling admission.
5. HTTP/2 keeps its separate decoded-header-list semantics and defaults; this proposal should not reinterpret the H2 limit as HTTP/1 wire bytes.
6. The implementation remains cancellation-safe and does not require allocating up to the current ~1 MiB ceiling when an adopter configures a substantially smaller budget.

If maintainers prefer these fields on a type other than `HttpServerOptions`, a per-connection/session option applied before `HttpSession::read_request()` would also satisfy the need as long as it is a supported public API.

## Describe alternatives you've considered

- **Reject in `ProxyHttp::request_filter()`:** useful for semantic policy, but too late for parser/allocation admission because `session.req_header()` already exists.
- **Place Nginx/another proxy in front:** defeats the purpose for deployments where Pingora is the edge authority and does not make Pingora's own parser budget configurable.
- **Fork/vendor `pingora-core` to change `MAX_HEADER_SIZE`/`MAX_HEADERS`:** enforces the right phase but creates a mutable supplier fork and makes upgrades/provenance harder.
- **Reuse HTTP/2 `max_header_list_size`:** not appropriate because HTTP/1 parser/wire accounting and H2 decoded header-list accounting are different contracts.

## Additional context

This request is based on source inspection of current protected `main` at `09696b51bc59315353d96686355861604d0bb48c`. A downstream integration issue tracking the same boundary is ContextualWisdomLab/pingora-gateway#43.

The goal is a supported resource-admission hook, not a product-specific header policy.

Contributor guide

Open the contributing guide

Research direction

Start with pingora-core/src/protocols/http/v1/common.rs and server.rs, then inspect HttpServerOptions and default_h2_options() for the existing configuration patterns. Trace normal and pipelined-prefix request parsing before deciding where supported byte and field-count limits belong. Done means unset limits preserve behavior, configured limits reject oversized requests before application callbacks, and HTTP/2 semantics remain separate.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
backend-api-design, networking
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.