apache / apache/apisix

Currently, all AI upstream services are simulated using this fake server method.

Open
#13,340 1 comment 0 reactions 0 assignees View on GitHub
tech debt
Dominant language
Lua
Stars
17.1k
Forks
2.9k
Avg merge
3d 16h
Merged PRs (30d)
63

Description

Currently, all AI upstream services are simulated using this fake server method.
I'm worried that the difference between the fake server and the real LLM request here is too big.

Should we introduce a container specifically for LLM Fake Server?

_Originally posted by @membphis in https://github.com/apache/apisix/pull/13307#discussion_r3201089261_

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the fake server method discussed in PR #13307 and compare its behavior with the real LLM request path. Determine the scope of a dedicated LLM fake-server container and validate that the simulated AI upstream services remain covered; completion should be a clear, tested decision and implementation plan.

Written by the indexing model from the issue text.

Assessment

Tech stack
lua
Domain
ai, testing
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.