Proposal: UDP/TCP header routing filters for dynamic backend routing
- Dominant language
- C++
- Stars
- 28.9k
- Forks
- 5.6k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 437
Description
*Title*: Add UDP/TCP header routing filters for dynamic backend routing
*Description*:
> This proposal adds two new Envoy filters that parse a custom 8-byte protocol header from UDP datagrams and TCP streams to dynamically route traffic to arbitrary backend servers (e.g. game battle rooms), without predefining static clusters. The room addresses are encoded in the header at runtime by the client and can change as containers scale.
*Use case*:
> Game battle servers deploy hundreds of rooms whose addresses are dynamically assigned by a container platform. Players are routed to an arbitrary room based on lobby assignment, and the room address is only known at runtime. This cannot be served by Envoy's static cluster model or existing UDP session filters (which only support source/destination IP/port matching, not payload inspection).
*Proposed design*:
> Two new filters share a single stateless parser:
> - `envoy.filters.udp.session.header_routing` (UDP session filter): parses the header from each datagram, strips it, and writes `envoy.upstream.dynamic_host` / `envoy.upstream.dynamic_port` filter state for the downstream `dynamic_forward_proxy` session filter.
> - `envoy.filters.network.header_routing` (TCP network filter): parses the header from the byte stream (with cross-segment buffering), strips it, and writes the same filter state for the downstream `sni_dynamic_forward_proxy` filter.
>
> Protocol header format (8 bytes, configurable magic/version):
> ```
> [Magic 1B][Version 1B][RoomIP 4B big-endian][RoomPort 2B big-endian]
> ```
>
> Both filters use the "block -> parse first packet -> continue filter chain" pattern validated against Envoy 1.39 source: `onNewSession()`/`onNewConnection()` returns `StopIteration` to block upstream selection; `onData()` parses the header, sets filter state, and calls `continueFilterChain()`/`continueReading()` to resume the filter chain so DFP reads the target address.
>
> A `forward_header` option (default true) controls whether the 8-byte header is forwarded to the upstream unchanged (true) or stripped (false).
*Alternatives considered*:
> 1. Predefine 500 clusters + use `envoy.udp_proxy.cluster` filter state: rejected because room addresses are dynamic and cannot be enumerated ahead of time.
> 2. Proxy-Wasm: rejected because Proxy-Wasm ABI has no UDP data-plane callbacks, and extending it would require upstream ABI/SDK changes across Envoy + Proxy-Wasm + SDK.
> 3. Dynamic Modules (Rust shared library): rejected because it would require reimplementing the full UDP forwarding path, bypassing `udp_proxy` and losing native session/LB/stats support.
> 4. Custom C++ session filter + DFP (this proposal): chosen because it fully reuses Envoy's native data-plane (sessions, LB, stats, timeouts) and `dynamic_forward_proxy` natively supports arbitrary dynamic IP:PORT.
*Extension policy compliance*:
> This adds two new extensions. Per GOVERNANCE.md extension addition policy, I'm opening this issue to discuss design before submitting a PR. The filters are stateless adapters around a shared pure-function parser, with full unit test coverage and an integration test suite.
*Design doc*:
> A detailed design document (including source-level timing analysis for Envoy 1.39, client contract, and risk assessment) is available at: https://github.com/wXwcoder/envoy/blob/pr/header-routing/udp_tcp_header_routing.md
*AI usage disclosure*:
> Generative AI was used to assist in writing the implementation, tests, and this proposal. The submitter has reviewed and fully understands all code and takes full ownership of the change.
Contributor guide
Research direction
Start with udp_tcp_header_routing.md and the referenced Envoy 1.39 source behavior; verify the proposed UDP session and TCP network filter extension points and dynamic_forward_proxy integration. Done means maintainers agree on the design and scope before implementation, with the parser, buffering, forwarding behavior, and unit and integration coverage defined.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- backend-api-design, networking
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100