apache / apache/apisix

help request: is ai-proxy plugin support multi endpoint in one plugin config?

Open
#13,453 2 comments 0 reactions 0 assignees View on GitHub
question
Dominant language
Lua
Stars
17.1k
Forks
2.9k
Avg merge
3d 16h
Merged PRs (30d)
63

Description

### Description

When using the ai-proxy plugin with the openai-compatible provider and configuring an override.endpoint, the endpoint must be a full URL including the path (http://127.0.0.1:8080/v1/chat/completions). because when I do not configure the URL in the endpoint of ai-proxy, the forwarding fails.

This creates a problem when you want to proxy multiple endpoints (e.g., /v1/chat/completions, /v1/embeddings, /v1/completions) from the same model service — you forced to:

1. Create multiple separate routes in APISIX, each with a different override.endpoint configuration, OR
2. Use ai-proxy-multi with multiple instances (which is designed for load balancing/fallback, not path-based routing)

### Environment

- APISIX version (run `apisix version`): 3.16.0
- Operating system (run `uname -a`): Linux 49e6a76a70b2 6.8.0-110-generic #110-Ubuntu SMP PREEMPT_DYNAMIC Thu Mar 19 15:09:20 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux

- OpenResty / Nginx version (run `openresty -V` or `nginx -V`):nginx version: openresty/1.27.1.2
built by gcc 13.3.0 (Ubuntu 13.3.0-6ubuntu2~24.04.1)
built with OpenSSL 3.4.1 11 Feb 2025
- etcd version, if relevant (run `curl http://127.0.0.1:9090/v1/server_info`):
- APISIX Dashboard version, if relevant:
- Plugin runner version, for issues related to plugin runners:
- LuaRocks version, for installation issues (run `luarocks --version`):

Contributor guide

Open the contributing guide

Research direction

No implementation files or tests are identified. Start by tracing the ai-proxy plugin's override.endpoint handling and compare it with ai-proxy-multi; define done as one plugin configuration forwarding the listed chat, embeddings, and completions paths to the same model service without separate routes.

Written by the indexing model from the issue text.

Assessment

Tech stack
lua
Domain
api, backend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.