Kong / Kong/developer.konghq.com
AI Gateway Passthrough mode
- Dominant language
- Ruby
- Stars
- 28
- Forks
- 121
- Avg merge
- 1d 4h
- Merged PRs (30d)
- 313
Description
## Overview
AI Gateway 2.1 introduces passthrough mode for AI traffic that Kong doesn't recognize or shouldn't transform.
This provides a supported way to proxy custom model servers, vendor preview APIs, and other AI endpoints that don't fit Kong's existing provider/format model, while preserving applicable Gateway capabilities.
## DoD
* Passthrough mode and its limitations are documented in a reference guide
* Compatibility with relevant AI Gateway capabilities is clear.
* Migration guidance from `route_type: preserve` is provided.
* Working **Konnect API** and **kongctl** examples are included.
AHA: https://konghq.aha.io/features/AI-63
----
## Scope
Document:
* What passthrough mode is and when to use it.
* How to configure it using **Konnect API** and **kongctl**.
* Request/response behavior and the fact that payloads aren't transformed.
* Available observability, including latency, status codes, and byte counts.
* Which authentication, rate limiting, guardrail, logging, and analytics capabilities are compatible with passthrough mode.
* Optional JSON path extraction for content and token usage, if included in the final 2.1 implementation.
* Migration from the deprecated `route_type: preserve`, including before/after configuration.
## Examples
Provide complete examples for:
* Basic passthrough to a custom AI endpoint.
* Passthrough with authentication/rate limiting.
* Content and token extraction, if shipped.
Examples should use the **AI Gateway 2.1 entity model**, not AI Proxy Advanced configuration.
## Notes
Research the best placement as part of this ticket. This will likely live alongside the relevant **AI Model / AI Model Provider** documentation rather than under AI Proxy Advanced.
PRD: https://docs.google.com/document/d/1MVKwkDY6Y2ZlvrOWlV44XXoRsBX7QAd_zXZr9cKTVXY/edit?tab=t.0#heading=h.1vtkg4pw82fb
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.