vercel / vercel/workflow

Step-dispatch latency grows linearly with accumulated run history (~0.3s per round per transition) — 65% of a 311-step run's wall time

Open
#3,026 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
2.4k
Forks
365
Avg merge
2d 11h
Merged PRs (30d)
169

Description

We run @workflow/ai@5.0.0-beta.11 + workflow@5.0.0-beta.24 on Vercel (IAD1). On long agent runs, the time between step completions grows linearly with the number of accumulated steps, even for trivial bookkeeping steps.

Measurement (a 311-step, 92-minute agent run on a preview deployment, all numbers from step createdAt/startedAt/completedAt timestamps via the workflow API):

  • Every step's own execution is fast: non-LLM steps complete in 0.1–0.5s once started; queue delay (startedAt − createdAt) ≈ 0 for all 311 steps.
  • But the per-TRANSITION gap (previous step end → next step created) grows from ~1.9s avg (round 2) to ~16.6s avg (round 50). Linear fit across rounds: gap = 1.45s + 0.297s × round, R² = 0.94. With ~6 transitions per agent round that's ~96s of pure engine overhead per round by round 50.
  • Budget: of 70.6 min of active run time, only 24.7 min (35%) is inside LLM steps; 45.9 min (65%) is inter-step time, 97.9% of which is not attributable to any step's execution or queue delay.
  • The growth is uniform across step types — a 0.1s credit-deduction step pays the same ~15s transition cost as an LLM step by round 50 — which points at per-dispatch run-history replay/rehydration cost rather than anything payload-specific.
  • Context: our conversation payloads are large (268K→589K tokens across the run), so O(history) work that re-reads/deserializes accumulated step results would scale badly here. We already strip the request.body echo from doStreamStep returns via a local patch (the core of #1932 / #1929) — this measurement is with that strip active.
  • History: we reported the same shape in April at much smaller magnitude (961ms→2,135ms over 77 steps) and #1338 flattened it to 0.27–0.47s. It has returned at ~10× on longer runs.

Our own mitigation (collapsing per-round bookkeeping steps) reduced dispatches per round from 6–7 to 3, which helps proportionally — but the per-dispatch growth itself looks engine-side.

Question: is per-dispatch cost expected to scale with accumulated run history in the current scheduler, and is there a knob or planned fix? Happy to share the full 311-step timing table and run IDs privately.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the workflow API timing fields (createdAt, startedAt, completedAt) and the doStreamStep return path; compare dispatch latency as accumulated run history grows using the 311-step measurements and prior issues #1338, #1932, and #1929. Done means determining whether scheduler replay or rehydration causes the linear cost and documenting a current knob or planned fix.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
backend, distributed-systems, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.