vllm-project / vllm-project/aibrix

Integrate the benchmark code into CI Job

Open
#1,050 0 comments 1 reaction 0 assignees View on GitHub
area/benchmark area/cicd priority/critical-urgent
Dominant language
Go
Stars
5.1k
Forks
697
Avg merge
1d 19h
Merged PRs (30d)
104

Description

### 🚀 Feature Description and Motivation

We've seen many issues related to benchmark
- https://github.com/vllm-project/aibrix/issues/1040
- https://github.com/vllm-project/aibrix/issues/1029
- https://github.com/vllm-project/aibrix/issues/1028

Can we integrate the code to CI system to make sure the code can run successfully? This at least give some confidence.

We can start with sample app and here's an example https://github.com/vllm-project/aibrix/blob/main/.github/workflows/installation-tests.yml

### Use Case

Improve the benchmark quality

### Proposed Solution

_No response_

Contributor guide

Open the contributing guide

Research direction

Start by reading .github/workflows/installation-tests.yml and the benchmark-related issues 1040, 1029, and 1028 to understand the sample app and existing failures. Identify the benchmark entry point from that context, then make the sample benchmark run in CI; done means the job executes successfully and provides confidence that the benchmark code works.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions, go
Domain
ci-cd, performance, testing-qa
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.