abetlen / abetlen/llama-cpp-python
Increase server timeout for long requests
Đang mở
enhancement
- Ngôn ngữ chính
- Python
- Star
- 10.6k
- Fork
- 1.4k
- Chỉ số merge pull request
- Chỉ số pull request đang chờ
Mô tả
While operating on the CPU, I am able to observe both the request and completion logs. However, due to the time-consuming nature of the request, the server issues a 504 timeout error before the completion is finalized. Therefore, I kindly propose the inclusion of an ENV variable or a flag that would allow for the adjustment of the server timeout duration.
Hướng dẫn đóng góp
Đánh giá
Issue này chưa được đánh giá.