feat: more precise control over the timeout for requests to the LLM service
- Dominant language
- Lua
- Stars
- 17.1k
- Forks
- 2.9k
- Avg merge
- 3d 16h
- Merged PRs (30d)
- 63
Description
### Description
Currently, AI-related plugins only have a timeout parameter. After reviewing the code, I found that timeout is the total of connect, read, and send timeout durations, making it impossible to precisely control the timeout for requests to the LLM service. This needs to be improved.
Currently https://github.com/apache/apisix/blob/master/apisix/plugins/ai-drivers/openai-base.lua#L70
```lua
...
local http = require("resty.http")
...
httpc:set_timeout(conf.timeout)
```
Expected:
```lua
...
local http = require("resty.http")
...
httpc:set_timeouts(connect_timeout, send_timeout, read_timeout)
```
Contributor guide
Research direction
Start with apisix/plugins/ai-drivers/openai-base.lua around the current httpc:set_timeout(conf.timeout) call, then trace how the AI plugin timeout configuration reaches resty.http. Check the existing AI plugin coverage before making the change. Done means requests can use separate connect, send, and read timeout values as described in the issue.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- lua
- Domain
- ai, api, backend
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100