Support isolated LLM Gateway for large request buffer policies

Problem

Large OpenAI-compatible /v1/chat/completions requests can exceed Envoy Gateway's default downstream connection buffer limit before the request reaches Envoy AI Gateway or vLLM. Envoy AI Gateway examples raise ClientTrafficPolicy.spec.connection.bufferLimit to 50Mi for AI workloads.

In a Nebari deployment where nebari-llm-serving is configured to patch llm-https / llm-internal-https listeners onto a shared platform Gateway, the practical workaround may be to attach that larger buffer policy to the entire shared Gateway. That fixes LLM requests, but it also changes downstream buffering behavior for unrelated services behind the same Gateway, such as landing pages, auth services, admin UIs, or other routes.

Why this matters

This is acceptable as a short-term deployment workaround, but it is not ideal as the long-term pattern. Buffer limits are connection/listener-level Envoy behavior, so deployments that need larger LLM request bodies should be able to isolate that behavior from non-LLM traffic.

Desired outcome

Provide or document a first-class deployment mode where LLM traffic uses a dedicated Gateway / Envoy entrypoint, so operators can safely apply LLM-specific client traffic policies such as:

apiVersion: gateway.envoyproxy.io/v1alpha1
kind: ClientTrafficPolicy
spec:
  targetRefs:
    - group: gateway.networking.k8s.io
      kind: Gateway
      name: <dedicated-llm-gateway>
  connection:
    bufferLimit: 50Mi

without affecting unrelated traffic on the shared platform Gateway.

Notes

The chart already exposes platform.gateway.external and platform.gateway.internal, and the operator reads those values via LLM_EXTERNAL_GATEWAY_* / LLM_INTERNAL_GATEWAY_*. The missing piece is a clear supported pattern, docs, or optional chart-managed resources for creating and wiring a dedicated LLM Gateway, including DNS/certificate expectations.

Start with the chart values platform.gateway.external and platform.gateway.internal, then trace how the operator reads LLM_EXTERNAL_GATEWAY_* and LLM_INTERNAL_GATEWAY_*. Define a supported dedicated LLM Gateway pattern or documentation, including wiring, DNS and certificate expectations, and isolation for ClientTrafficPolicy buffer limits. Done means operators can apply LLM-specific buffering without changing unrelated traffic on the shared Gateway.