vllm-project / vllm-project/aibrix

[Feature] Add session-aware Agent workload generation

Open
#2,482 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
5.1k
Forks
697
Avg merge
1d 19h
Merged PRs (30d)
104

Description

## Motivation

AIBrix workload generation currently models requests primarily through arrival times and prompt/output lengths. Agent applications add session-level structure: multiple model calls, parallel tools, retries, waits, joins, context growth, and state reuse. Without that structure, routing, cache-locality, autoscaling, and capacity experiments cannot reproduce Agent bursts and correlated requests.

## Proposal

Add an opt-in `agent` trace type to the existing workload generator. It should:

- generate deterministic session DAGs with model, tool, retry, wait, and join nodes
- project model nodes into the existing `timestamp` + `requests` format so the current benchmark client remains compatible
- expose non-model structure through an additive `events` field
- use anonymous deterministic identifiers and synthetic prompts only
- allow configuration of session arrival rate, turns, parallel tools, latency ranges, retries, context growth, output length, and seed
- include unit tests and an example configuration

## Compatibility

Existing trace types and clients remain unchanged. Clients that only read `timestamp` and `requests` ignore the additional graph metadata.

## Intended use

The generated trace enables session-affinity, prefix-cache, burst, and Agent capacity experiments without including production content or reversible user identifiers.

Contributor guide

Open the contributing guide

Research direction

Start by locating the existing workload generator and trace-type handling, then inspect how timestamp and requests are consumed by the benchmark client. Define unit tests for deterministic session DAGs, projection compatibility, and additive events, along with the requested example configuration. Done means existing trace types remain unchanged and the new agent trace supports the listed configuration options.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
performance, tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.