vllm-project / vllm-project/aibrix
Support debug port in gateway plugin
Open
- Dominant language
- Go
- Stars
- 5.1k
- Forks
- 694
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 98
Description
### 🚀 Feature Description and Motivation
service discovery is commonly misconfigured by users. However, we do not have every straightforward ways to help user diagnose the problem.
The most easiest way I can come up with is to provide a debug interface and expose the endpoint information. for deployment, model to pod list mapping is good. lora is same. for StormService, we need to list the roles
### Use Case
gateway plugin
### Proposed Solution
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.