help request: openai-base.lua:366: failed to connect to LLM server: timeout
- Dominant language
- Lua
- Stars
- 17.1k
- Forks
- 2.9k
- Avg merge
- 3d 16h
- Merged PRs (30d)
- 63
Description
### Description
I am deployed Apache APISIX on my Ubuntu system and using the ai-proxy plugin as an LLM router. During operation, I encountered an intermittent problem: "openai-base.lua:366: failed to connect to LLM server: timeout" appears periodically, but then recovers after some time. I configured various configurations from different LLM manufacturers in the routing list, including Doubao, Qwen, and DeepSeek. This problem occurred for each manufacturer's model. However, directly using the official base URL provided by the LLM vendors works without any problems.
After extensive analysis, I haven't been able to identify the root cause. Any help from the community would be greatly appreciated!
### Environment
- APISIX version (run `apisix version`): 3.16.0
- Operating system (run `uname -a`): Ubuntu 24.04.3
- OpenResty / Nginx version (run `openresty -V` or `nginx -V`): openresty/1.27.1.2
- etcd version, if relevant (run `curl http://127.0.0.1:9090/v1/server_info`): etcd Version: 3.5.4
- APISIX Dashboard version, if relevant:
- Plugin runner version, for issues related to plugin runners:
- LuaRocks version, for installation issues (run `luarocks --version`):
Contributor guide
Research direction
Start at openai-base.lua:366 in the ai-proxy plugin and trace how routed requests connect to the configured LLM servers. Compare the plugin's requests with direct requests to the vendors' official base URLs, using the reported APISIX, OpenResty, Ubuntu, and etcd versions. Done means identifying the intermittent timeout cause and documenting or fixing the affected configuration or request path.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- lua
- Domain
- api, backend-api-design, networking
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100