Experiment: workflow-aware prefix residency plus request admission for multi-agent serving

Finding

TOPAS, arXiv:2608.25523, submitted 2026-08-26, treats agent-prefix KV residency and request admission as one workflow-level scheduling decision rather than separate cache and request policies.

The originating team implemented TOPAS in SGLang v0.5.3 on a single NVIDIA A100 80GB with Qwen2.5-32B-Instruct. Against the strongest baseline per workload and metric, they report mean/p99 task completion time reductions up to 39.8/49.4 percent on synthetic DAGs, 9.8 percent mean reduction on MetaGPT-SOP, and 22.0/26.6 percent mean/p99 reduction on MetaGPT-TL. In a MetaGPT-SOP overhead run at 0.15 task/s, the scheduler averaged 1.9 ms per decision and 0.31 percent of wall time.

Evidence class: originating-team measured systems result, not independently reproduced by RuV.

RuV hypothesis

Ruflo currently sees workflow structure and agent progress while serving layers see prefix/KV state. The missing interface may be a small ServingStateReceipt, not a new full scheduler.

Expose from orchestration to the serving adapter:

• task/workflow identity
• ready agent stages
• longest remaining dependency path estimate
• agent prefix identity and token length
• current prefix residency
• dynamic KV reservation estimate
• expected near-term prefix reuse
• task age
• preemption/reload cost estimate

Then benchmark whether a workflow-aware admission policy can jointly choose prefix residency and ready requests more efficiently than request-local scheduling.

Conditions

A. Current serving scheduler.
B. Locality-first prefix policy.
C. Progress-first workflow policy.
D. Joint workflow-aware prefix plus admission policy.

Use the same model, SGLang version, GPU, task arrivals, workflow traces, prompts, KV budget, and seeds for all conditions.

Workloads

Include:

• one linear multi-agent pipeline
• one fork/join DAG
• one recurrent supervisor/specialist topology
• one real Ruflo coding or research workflow
• burst arrivals and low-load arrivals
• tool waits and variable ReAct iteration counts
• memory pressure causing prefix movement

Metrics

• mean/p95/p99 task completion time
• request TTFT and inter-token latency
• throughput and goodput
• prefix hit/reuse rate
• prefix movement bytes and time
• preemption/reconstruction work
• GPU KV occupancy
• batch size
• starvation incidents
• scheduler p50/p95 latency
• GPU-hours and energy if available
• task success and failures

Required falsification

Test whether simple longest-prefix matching plus task aging performs within variance. Test low-load conditions where TOPAS-like state search may add complexity without benefit. Test highly dynamic workflows where remaining-path estimates are wrong. Record any workload where the joint policy loses.

Promotion gate

Advance only if the candidate improves mean or p99 task completion time by at least 15 percent on a real Ruflo workflow at matched task success, while scheduler overhead remains below 1 percent of wall time and no starvation or tail-latency regression above 5 percent appears on protected workloads.

If gains occur only on synthetic DAGs, keep the result experimental.

Integration and governance

Ruflo owns workflow state; Cognitum/serving owns GPU placement and cache mechanics. Keep the receipt interface provider-neutral so SGLang is an adapter, not an architectural dependency. No scheduling confidence grants capability or data authority. No autonomous deployment.

Rollback

Disable the workflow-aware adapter and return to the current request scheduler. No model or persistent state migration.

The issue names no repository files, tests, or entry points. Start by locating Ruflo's workflow orchestration and serving-adapter boundaries, then compare the current scheduler with the four proposed conditions. Done means a provider-neutral receipt, matched benchmarks across the listed workloads and metrics, falsification results, and a rollback path if the promotion gate is not met.