envoyproxy / envoyproxy/envoy

Proposal: UDP/TCP header routing filters for dynamic backend routing

Open
#47,147 1 comment 0 reactions 0 assignees View on GitHub
area/dynamic_forward_proxy area/extension area/udp enhancement
Dominant language
C++
Stars
28.9k
Forks
5.6k
Avg merge
1d 20h
Merged PRs (30d)
437

Description

*Title*: Add UDP/TCP header routing filters for dynamic backend routing

*Description*:
> This proposal adds two new Envoy filters that parse a custom 8-byte protocol header from UDP datagrams and TCP streams to dynamically route traffic to arbitrary backend servers (e.g. game battle rooms), without predefining static clusters. The room addresses are encoded in the header at runtime by the client and can change as containers scale.

*Use case*:
> Game battle servers deploy hundreds of rooms whose addresses are dynamically assigned by a container platform. Players are routed to an arbitrary room based on lobby assignment, and the room address is only known at runtime. This cannot be served by Envoy's static cluster model or existing UDP session filters (which only support source/destination IP/port matching, not payload inspection).

*Proposed design*:
> Two new filters share a single stateless parser:
> - `envoy.filters.udp.session.header_routing` (UDP session filter): parses the header from each datagram, strips it, and writes `envoy.upstream.dynamic_host` / `envoy.upstream.dynamic_port` filter state for the downstream `dynamic_forward_proxy` session filter.
> - `envoy.filters.network.header_routing` (TCP network filter): parses the header from the byte stream (with cross-segment buffering), strips it, and writes the same filter state for the downstream `sni_dynamic_forward_proxy` filter.
>
> Protocol header format (8 bytes, configurable magic/version):
> ```
> [Magic 1B][Version 1B][RoomIP 4B big-endian][RoomPort 2B big-endian]
> ```
>
> Both filters use the "block -> parse first packet -> continue filter chain" pattern validated against Envoy 1.39 source: `onNewSession()`/`onNewConnection()` returns `StopIteration` to block upstream selection; `onData()` parses the header, sets filter state, and calls `continueFilterChain()`/`continueReading()` to resume the filter chain so DFP reads the target address.
>
> A `forward_header` option (default true) controls whether the 8-byte header is forwarded to the upstream unchanged (true) or stripped (false).

*Alternatives considered*:
> 1. Predefine 500 clusters + use `envoy.udp_proxy.cluster` filter state: rejected because room addresses are dynamic and cannot be enumerated ahead of time.
> 2. Proxy-Wasm: rejected because Proxy-Wasm ABI has no UDP data-plane callbacks, and extending it would require upstream ABI/SDK changes across Envoy + Proxy-Wasm + SDK.
> 3. Dynamic Modules (Rust shared library): rejected because it would require reimplementing the full UDP forwarding path, bypassing `udp_proxy` and losing native session/LB/stats support.
> 4. Custom C++ session filter + DFP (this proposal): chosen because it fully reuses Envoy's native data-plane (sessions, LB, stats, timeouts) and `dynamic_forward_proxy` natively supports arbitrary dynamic IP:PORT.

*Extension policy compliance*:
> This adds two new extensions. Per GOVERNANCE.md extension addition policy, I'm opening this issue to discuss design before submitting a PR. The filters are stateless adapters around a shared pure-function parser, with full unit test coverage and an integration test suite.

*Design doc*:
> A detailed design document (including source-level timing analysis for Envoy 1.39, client contract, and risk assessment) is available at: https://github.com/wXwcoder/envoy/blob/pr/header-routing/udp_tcp_header_routing.md

*AI usage disclosure*:
> Generative AI was used to assist in writing the implementation, tests, and this proposal. The submitter has reviewed and fully understands all code and takes full ownership of the change.

Contributor guide

Open the contributing guide

Research direction

Start with udp_tcp_header_routing.md and the referenced Envoy 1.39 source behavior; verify the proposed UDP session and TCP network filter extension points and dynamic_forward_proxy integration. Done means maintainers agree on the design and scope before implementation, with the parser, buffering, forwarding behavior, and unit and integration coverage defined.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
backend-api-design, networking
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.