Envoy not freeing memory even after connections and requests decrease

We observe that Envoy does not seem to release memory back to the system (as quickly as the connections go down). In my case it's about Istio Gateways (1.27.2) which we auto-scale on memory consumption (taken from Kubelet / CAdvisor metrics via HPA). Scaling up works just fine this way as memory nicely correlates with the increase of connections and rps. But scaling down does not work as well as memory is only released very slowly (if even) compared to the observed drop in concurrent connections and rps:

Connections


Request Rate


Memory Consumption in GB:



And we also run with a max connection duration of 15 minutes, so there are no "old" and idle connections.

_Originally posted by @frittentheke in [#39006](https://github.com/envoyproxy/envoy/issues/39006#issuecomment-3419867027)_ which was about another memory not freed issue ...