Step-dispatch latency grows linearly with accumulated run history (~0.3s per round per transition) — 65% of a 311-step run's wall time
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 2.4k
- Forks
- 365
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 169
Description
We run @workflow/ai@5.0.0-beta.11 + workflow@5.0.0-beta.24 on Vercel (IAD1). On long agent runs, the time between step completions grows linearly with the number of accumulated steps, even for trivial bookkeeping steps.
Measurement (a 311-step, 92-minute agent run on a preview deployment, all numbers from step createdAt/startedAt/completedAt timestamps via the workflow API):
- Every step's own execution is fast: non-LLM steps complete in 0.1–0.5s once started; queue delay (startedAt − createdAt) ≈ 0 for all 311 steps.
- But the per-TRANSITION gap (previous step end → next step created) grows from ~1.9s avg (round 2) to ~16.6s avg (round 50). Linear fit across rounds: gap = 1.45s + 0.297s × round, R² = 0.94. With ~6 transitions per agent round that's ~96s of pure engine overhead per round by round 50.
- Budget: of 70.6 min of active run time, only 24.7 min (35%) is inside LLM steps; 45.9 min (65%) is inter-step time, 97.9% of which is not attributable to any step's execution or queue delay.
- The growth is uniform across step types — a 0.1s credit-deduction step pays the same ~15s transition cost as an LLM step by round 50 — which points at per-dispatch run-history replay/rehydration cost rather than anything payload-specific.
- Context: our conversation payloads are large (268K→589K tokens across the run), so O(history) work that re-reads/deserializes accumulated step results would scale badly here. We already strip the
request.bodyecho from doStreamStep returns via a local patch (the core of #1932 / #1929) — this measurement is with that strip active. - History: we reported the same shape in April at much smaller magnitude (961ms→2,135ms over 77 steps) and #1338 flattened it to 0.27–0.47s. It has returned at ~10× on longer runs.
Our own mitigation (collapsing per-round bookkeeping steps) reduced dispatches per round from 6–7 to 3, which helps proportionally — but the per-dispatch growth itself looks engine-side.
Question: is per-dispatch cost expected to scale with accumulated run history in the current scheduler, and is there a knob or planned fix? Happy to share the full 311-step timing table and run IDs privately.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the workflow API timing fields (createdAt, startedAt, completedAt) and the doStreamStep return path; compare dispatch latency as accumulated run history grows using the 311-step measurements and prior issues #1338, #1932, and #1929. Done means determining whether scheduler replay or rehydration causes the linear cost and documenting a current knob or planned fix.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- backend, distributed-systems, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100