Prompt compression and context-window-aware truncation

Description

Nothing compresses a prompt toward a token budget, and nothing reacts to the model context window: there is no truncation of old turns, no rolling summarisation, no message dedup, no semantic compression, and no policy accepting a target token budget or a context-limit ratio.

Requested: such a policy, with

• configurable strategies — drop oldest turns, summarise the middle, dedupe repeated context, strip boilerplate;
• tokenizer-accurate per-model accounting;
• a deterministic path that costs no extra model call;
• metrics for tokens saved and compressions applied.

Why

Token spend is the dominant cost line in production LLM use, and long agent conversations fail outright with context-length errors — a failure the gateway is uniquely placed to prevent, since it already parses and counts every message on the path.

Priority

Medium.

Prior art

LiteLLM and Kong both ship budget-targeted compression today.

_{Surveyed 2026-07-15; every claim rests on a fetched docs/source page.}