vllm-project / vllm-project/guidellm

Add support for vLLMs new gRPC interface

Open
#548 1 comment 0 reactions 1 assignee View on GitHub

@dbutenhof is already working on this.

Since Mar 3, 2026.

priority-medium
Dominant language
Python
Stars
1.6k
Forks
234
Avg merge
2d 22h
Merged PRs (30d)
46

Description

In vLLM 0.14.0+, there is now first-class support for a gRPC set of endpoints. It would be great to have the ability to have guidellm optionally use this interface instead of HTTP/S for traffic, I would expect this to be a mutually exclusive mode.

Link to proto: https://github.com/vllm-project/vllm/blob/69d09fdd6cb66fdfc92f2f2388c4899dfd93d0ad/vllm/grpc/vllm_engine.proto#L2

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.