Kong / Kong/developer.konghq.com

AI Gateway Passthrough mode

Open
#6,764 1 comment 0 reactions 1 assignee Claimed by @lmilan View on GitHub
internal product:ai-gateway
Dominant language
Ruby
Stars
28
Forks
121
Avg merge
1d 4h
Merged PRs (30d)
313

Description

## Overview

AI Gateway 2.1 introduces passthrough mode for AI traffic that Kong doesn't recognize or shouldn't transform.

This provides a supported way to proxy custom model servers, vendor preview APIs, and other AI endpoints that don't fit Kong's existing provider/format model, while preserving applicable Gateway capabilities.

## DoD
* Passthrough mode and its limitations are documented in a reference guide
* Compatibility with relevant AI Gateway capabilities is clear.
* Migration guidance from `route_type: preserve` is provided.
* Working **Konnect API** and **kongctl** examples are included.

AHA: https://konghq.aha.io/features/AI-63

----
## Scope

Document:

* What passthrough mode is and when to use it.
* How to configure it using **Konnect API** and **kongctl**.
* Request/response behavior and the fact that payloads aren't transformed.
* Available observability, including latency, status codes, and byte counts.
* Which authentication, rate limiting, guardrail, logging, and analytics capabilities are compatible with passthrough mode.
* Optional JSON path extraction for content and token usage, if included in the final 2.1 implementation.
* Migration from the deprecated `route_type: preserve`, including before/after configuration.

## Examples

Provide complete examples for:

* Basic passthrough to a custom AI endpoint.
* Passthrough with authentication/rate limiting.
* Content and token extraction, if shipped.

Examples should use the **AI Gateway 2.1 entity model**, not AI Proxy Advanced configuration.

## Notes

Research the best placement as part of this ticket. This will likely live alongside the relevant **AI Model / AI Model Provider** documentation rather than under AI Proxy Advanced.

PRD: https://docs.google.com/document/d/1MVKwkDY6Y2ZlvrOWlV44XXoRsBX7QAd_zXZr9cKTVXY/edit?tab=t.0#heading=h.1vtkg4pw82fb

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.