envoyproxy / envoyproxy/envoy

[ext_proc] add a control message directs modes overrides before request header response

Open
#46,125 6 comments 0 reactions 0 assignees View on GitHub
area/ext_proc enhancement
Dominant language
C++
Stars
28.9k
Forks
5.6k
Avg merge
1d 22h
Merged PRs (30d)
430

Description

*Title*: [ext_proc] add a control message directs modes overrides before request header response

*Description*:
In MT cloud proxy, oftentimes proxy itself doesn't want to buffer the whole body payload due to RAM pressure. Esp so when AI traffic could easily be up to MBs in size. Modes allows in ext_proc are usually streamed or FULL_DUPLEX_STREAMED.

A feature frequently asked by customer: I want to see the headers, based on it I can make decision to either reject it or I will want body for further inspection, but **DONOT send headers to upstream yet**.

STREAMED mode would cover the part of the story: the NO path
1. send headers to ext_proc server;
1. ext_proc server sends back headers response, and mode override set to None; with a immediate response.
1. immediate response goes back to sender.

The YES path is a little bit trickier tho: (STREAMED)
1. send headers to ext_proc server;
1. ext_proc server sends back headers response, and mode override set to FULL_DUPLEX_STREAMED
1. Data chunks sent to ext_proc for further inspect, ext_proc server sends immdiate_response.

Streamed mode break the expectation at step 3.: Headers response when received, will be forwarded to upstream already.

FULL_DUPLEX_STREAMED mode doesn't work neither: body chunk streaming starts not waiting for headers responses.

Let's add a control-message, very similar to how override_message_timeout is done today, to FULL_DUPLEX_STREAMED mode (or more modes), that can apply mode-overides/timeout/other config application from ext_proc server, before any header response begins.

this way we can answer what's asked by the customers:
1. in streamed mode, ext_proc sends back the control message: mode switch to full_duplex_streamed;
2. no header response received yet, body sent to ext_proc server;
3. ext_proc server sends back immediate response, or normal header responses, after check the headers AND the body.

Contributor guide

Open the contributing guide

Research direction

Start by tracing the existing ext_proc handling for override_message_timeout, then follow the streamed and FULL_DUPLEX_STREAMED response sequencing. Done means a control message can switch modes before header responses are forwarded, permit body inspection, and preserve the immediate or normal response paths described.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
networking
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.