vllm-project / vllm-project/aibrix
[Feature] Add session-aware Agent workload generation
- Dominant language
- Go
- Stars
- 5.1k
- Forks
- 697
- Avg merge
- 1d 19h
- Merged PRs (30d)
- 104
Description
## Motivation
AIBrix workload generation currently models requests primarily through arrival times and prompt/output lengths. Agent applications add session-level structure: multiple model calls, parallel tools, retries, waits, joins, context growth, and state reuse. Without that structure, routing, cache-locality, autoscaling, and capacity experiments cannot reproduce Agent bursts and correlated requests.
## Proposal
Add an opt-in `agent` trace type to the existing workload generator. It should:
- generate deterministic session DAGs with model, tool, retry, wait, and join nodes
- project model nodes into the existing `timestamp` + `requests` format so the current benchmark client remains compatible
- expose non-model structure through an additive `events` field
- use anonymous deterministic identifiers and synthetic prompts only
- allow configuration of session arrival rate, turns, parallel tools, latency ranges, retries, context growth, output length, and seed
- include unit tests and an example configuration
## Compatibility
Existing trace types and clients remain unchanged. Clients that only read `timestamp` and `requests` ignore the additional graph metadata.
## Intended use
The generated trace enables session-affinity, prefix-cache, burst, and Agent capacity experiments without including production content or reversible user identifiers.
Contributor guide
Research direction
Start by locating the existing workload generator and trace-type handling, then inspect how timestamp and requests are consumed by the benchmark client. Define unit tests for deterministic session DAGs, projection compatibility, and additive events, along with the requested example configuration. Done means existing trace types remain unchanged and the new agent trace supports the listed configuration options.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- performance, tooling
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100