Envoy consumes high memory when Ext Proc is used with the FULL_DUPLEX_STREAMED mode

**If you are reporting *any* crash or *any* potential security issue, *do not*
open an issue in this repo. Please report the issue via emailing
envoy-security@googlegroups.com where the issue will be triaged appropriately.**

*Title*: *Envoy consumes high memory when Ext Proc is used with the FULL_DUPLEX_STREAMED mode*

*Description*:
- Envoy Version: v1.34.1
- Please find my POC: https://github.com/renuka-fernando/envoy-ext-proc-body-streaming

My use case involves transforming both the request and response bodies. Consider the following scenario:

A client uploads an MP4 file (70.4 MiB), and the Ext Proc server transforms it into a new MP4 file (78 MiB) before forwarding it to the upstream service. The response body undergoes a similar transformation. For this POC, the new body is read from the file system.

- One key requirement is that the Ext Proc server must read the entire request body before sending the transformed version. The same applies to the response body.


The Ext Proc service reads the request body in chunks and writes it to the file system to verify that the entire body has been received. Once the full request body is read, it loads the new body from the file system and replaces the original request body. The same process is applied to the response body. This approach worked flawlessly without any issues.

#### FULL_DUPLEX_STREAMED mode (700 MiB)

However, when I sent 10 concurrent requests, each with a 70.4 MiB payload, Envoy’s memory usage rose to around 500 MiB. Slowing down the Ext Proc service caused Envoy to consume even more memory—approximately 700 MiB. You can replicate this in my POC by increasing the sleep duration to 10 seconds.

#### STREAMED mode (150 MiB)

I also experimented with the STREAMED mode. In this configuration, the Ext Proc service updates the response chunk as soon as it receives a request chunk. It reads the first chunk of the request and immediately replies with a new response chunk. This breaks my requirement, as I need to read the entire request body before sending any part of the response body. In this scenario, Envoy’s memory usage peaked at around 150 MiB. Even when I slowed down the process, it only reached a maximum of 110 MiB.

When I checked the Envoy code, I noticed the following.
#### In STREAMED mode:
- https://github.com/envoyproxy/envoy/blob/v1.34.1/source/extensions/filters/http/ext_proc/processor_state.cc#L103
- A request message is constructed and queued for the Ext Proc service.
- The message is then added to an internal queue.
- https://github.com/envoyproxy/envoy/blob/v1.34.1/source/extensions/filters/http/ext_proc/processor_state.cc#L529
- Envoy raises the watermark to stop reading additional request body data from downstream.
- Once a response is received from the Ext Proc, the watermark is lowered, and downstream reading resumes.
#### In FULL_DUPLEX_STREAMED mode:
- https://github.com/envoyproxy/envoy/blob/v1.34.1/source/extensions/filters/http/ext_proc/ext_proc.cc#L611
- Data is read from downstream, and a corresponding request is created for the Ext Proc.
- The request is drained, and downstream reading continues without raising the watermark.
- If the Ext Proc processes data slowly, Envoy continues to generate request objects containing payload chunks, which are buffered until consumed.
- This behavior leads to increased memory usage in Envoy.

[optional *Relevant Links*:]

https://github.com/renuka-fernando/envoy-ext-proc-body-streaming