cloudflare / cloudflare/pingora
Expose configurable HTTP/1 request-header parser admission limits
- Dominant language
- Rust
- Stars
- 27.4k
- Forks
- 1.7k
- Avg merge
- 6h 22m
- Merged PRs (30d)
- 3
Description
## What is the problem your feature solves, or the need it fulfills?
At current `main` (`09696b51bc59315353d96686355861604d0bb48c`), downstream HTTP/1 request-header admission is fixed inside `pingora-core`:
- `pingora-core/src/protocols/http/v1/common.rs` defines `MAX_HEADERS = 256`, `INIT_HEADER_BUF_SIZE = 4096`, and `MAX_HEADER_SIZE = 1_048_575`.
- `HttpSession::read_request()` in `pingora-core/src/protocols/http/v1/server.rs` grows its parser buffer against that fixed ceiling. The source notes that the size check happens before the next read, so a completed large socket read can already exceed the nominal threshold before rejection.
- `HttpServerOptions` currently exposes transport options such as h2c, keepalive request limits, and H2 idle timeout, but no HTTP/1 request-header byte/count admission setting.
For an edge proxy that needs an operator-controlled header budget materially below the built-in ceiling, rejecting in a `ProxyHttp::request_filter()` callback is too late to serve as a parser/resource admission control: the request header has already been buffered and parsed before that callback.
HTTP/2 already has a different control surface: `default_h2_options()` applies a decoded header-list limit and `H2Options::max_header_list_size` can customize it. I am not proposing that HTTP/1 wire/parser bytes and HTTP/2 decoded header-list size be treated as identical accounting.
## Describe the solution you'd like
Please expose a supported HTTP/1 server/composition option that can lower request-header resource admission before or during parser buffering, without requiring downstream users to copy or patch Pingora parser source.
A useful API would preserve today's behavior when unset and allow an adopter to configure, at minimum:
- a positive maximum HTTP/1 request-header byte budget; and
- preferably a positive maximum HTTP/1 header-field count, bounded by the existing safe supplier maximum.
The limit should be applied by the HTTP/1 parser path itself, including pipelined-prefix handling, rather than only after `RequestHeader` construction. Oversized traffic may fail at the transport/parser boundary; a specific application status such as 431 is less important than enforcing the resource bound before application callbacks.
Expected compatibility/acceptance properties:
1. `None`/unset preserves the current constants and behavior.
2. Near-limit HTTP/1 requests succeed; over-limit one-large-field and many-small-fields requests fail before application callback processing.
3. The configured byte/count limits also apply when the request begins from a pipelined prefix on a reused connection.
4. Zero/impossible values fail configuration rather than silently disabling admission.
5. HTTP/2 keeps its separate decoded-header-list semantics and defaults; this proposal should not reinterpret the H2 limit as HTTP/1 wire bytes.
6. The implementation remains cancellation-safe and does not require allocating up to the current ~1 MiB ceiling when an adopter configures a substantially smaller budget.
If maintainers prefer these fields on a type other than `HttpServerOptions`, a per-connection/session option applied before `HttpSession::read_request()` would also satisfy the need as long as it is a supported public API.
## Describe alternatives you've considered
- **Reject in `ProxyHttp::request_filter()`:** useful for semantic policy, but too late for parser/allocation admission because `session.req_header()` already exists.
- **Place Nginx/another proxy in front:** defeats the purpose for deployments where Pingora is the edge authority and does not make Pingora's own parser budget configurable.
- **Fork/vendor `pingora-core` to change `MAX_HEADER_SIZE`/`MAX_HEADERS`:** enforces the right phase but creates a mutable supplier fork and makes upgrades/provenance harder.
- **Reuse HTTP/2 `max_header_list_size`:** not appropriate because HTTP/1 parser/wire accounting and H2 decoded header-list accounting are different contracts.
## Additional context
This request is based on source inspection of current protected `main` at `09696b51bc59315353d96686355861604d0bb48c`. A downstream integration issue tracking the same boundary is ContextualWisdomLab/pingora-gateway#43.
The goal is a supported resource-admission hook, not a product-specific header policy.
Contributor guide
Research direction
Start with pingora-core/src/protocols/http/v1/common.rs and server.rs, then inspect HttpServerOptions and default_h2_options() for the existing configuration patterns. Trace normal and pipelined-prefix request parsing before deciding where supported byte and field-count limits belong. Done means unset limits preserve behavior, configured limits reject oversized requests before application callbacks, and HTTP/2 semantics remain separate.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- backend-api-design, networking
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100