vercel / vercel/workflow

Stream event delivery to clients lags 2-3 minutes behind durable writes (steps finish in seconds)

Open
#2,767 3 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
2.4k
Forks
365
Avg merge
2d 11h
Merged PRs (30d)
169

Description

Summary

Events written to a workflow run's stream become visible to readers only minutes later. The workflow itself completes its steps in seconds, but a client holding GET .../stream receives the events in ~120s reconnect cycles, and even a fresh stream read (replay from startIndex=0) does not show the recorded events until ~90s after the step finished.

Environment

  • Vercel-managed Workflow (production, Fluid Compute), workflow spec v5
  • Reached through the eve framework, which bundles @workflow/core — reproduced identically on eve 0.11.10 and 0.18.1, so this does not appear to be an eve regression
  • Local dev runtime delivers the same streams in real time (<20s end-to-end), so this is specific to the hosted workflow backend

Measurements

Single structured-output turn; the model step completes in seconds — confirmed via /flow invocation logs.

Read strategy Time until result.completed reached the client
Hold one stream connection (default client) 124s / 179s / 199s (3 runs)
Fresh short-lived stream GETs every 3s, startIndex=0 replay 90s (17 polls before the event appeared)
Same code against local dev runtime ~15–20s

Request-log pattern for the held-connection case: POST /session 202 in ~1s → /.well-known/workflow/v1/flow invocations complete within seconds → stream GET reconnects at exactly +120s intervals, and the result arrives only on a later cycle. Setting WORKFLOW_V2_TIMEOUT_MS=30000 had no effect on delivery timing.

The replay experiment is the interesting datapoint: since a brand-new streams.get() read at +60s still did not include events recorded at ~+10s, the lag appears to be on the write-visibility side of the streams backend, not in client reconnect behavior.

Impact

Interactive use (chat-style refinement against a durable session) is effectively blocked: every turn costs 2–3 minutes of waiting for a result that was ready in seconds. Possibly related to #943 (functions hanging after steps complete) if the underlying cause is stream finalization.

Happy to provide session ids / timestamps privately for backend-side tracing.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the hosted behavior using the /flow invocation logs, a held GET .../stream connection, and fresh streams.get() reads with startIndex=0. Compare event-write and client-visible timestamps against the local runtime; done means recorded events become visible promptly instead of waiting through 90–120 second delays.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
backend, distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.