vllm-project / vllm-project/aibrix

Support debug port in gateway plugin

Open
#1,685 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
5.1k
Forks
694
Avg merge
1d 20h
Merged PRs (30d)
98

Description

### 🚀 Feature Description and Motivation

service discovery is commonly misconfigured by users. However, we do not have every straightforward ways to help user diagnose the problem.

The most easiest way I can come up with is to provide a debug interface and expose the endpoint information. for deployment, model to pod list mapping is good. lora is same. for StormService, we need to list the roles

### Use Case

gateway plugin

### Proposed Solution

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.