Currently, all AI upstream services are simulated using this fake server method.
- Dominant language
- Lua
- Stars
- 17.1k
- Forks
- 2.9k
- Avg merge
- 3d 16h
- Merged PRs (30d)
- 63
Description
Currently, all AI upstream services are simulated using this fake server method.
I'm worried that the difference between the fake server and the real LLM request here is too big.
Should we introduce a container specifically for LLM Fake Server?
_Originally posted by @membphis in https://github.com/apache/apisix/pull/13307#discussion_r3201089261_
Contributor guide
Research direction
Start by reviewing the fake server method discussed in PR #13307 and compare its behavior with the real LLM request path. Determine the scope of a dedicated LLM fake-server container and validate that the simulated AI upstream services remain covered; completion should be a clear, tested decision and implementation plan.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- lua
- Domain
- ai, testing
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 38/100