vercel / vercel/workflow

Workflow World returns 404 not-found for runs created by newly-deployed functions (@workflow/world-vercel)

Open
#2,735 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
2.4k
Forks
365
Avg merge
2d 11h
Merged PRs (30d)
169

Description

Summary

Since ~2026-07-01 08:57 UTC, workflow runs created by any newly-created deployment fail: writes to the managed Workflow World (vercel-workflow.com) return HTTP 404 not-found for a run the deployment just created. Runs created by an older deployment of the same project keep working, and rolling the production alias back to a pre-08:57-UTC deployment fully restored service. The application code and dependency lockfile are byte-identical across the working and failing deployments, so this looks like a platform-side Workflow World registration/provisioning problem tied to newly-created deployments rather than an app change.

A Vercel support case has been filed with the full private details (team/project/deployment IDs): Support ticket 01O1qb7pAz2Gs2tP.

Environment
  • @workflow/core 5.0.0-beta.26
  • @workflow/world-vercel 5.0.0-beta.22
  • workflow 5.0.0-beta.26
  • Runtime: bun1.x, Nitro (preset vercel), compatibility date 2026-07-01
  • Region observed: fra1
  • Deployed via source build (bun install --ignore-scripts on Vercel; lockfile unchanged between working/failing builds)
Primary error (repeats for essentially every new run)
[workflow-sdk] Background flush of step stream ops failed
  run:  wrun_01KWEF3VV779FRP13S88W078ZN
  step: step_01KWEF3VV7E0FPGZESQWQGK8W2
  error: Stream write failed: HTTP 404
  PUT https://vercel-workflow.com/api/v2/runs/wrun_01KWEF3VV779FRP13S88W078ZN/stream/strm_01KWEF3VV779FRP13S88W078ZN_user
  response: {"success":false,"error":"not-found","message":"workflow run wrun_01KWEF3VV779FRP13S88W078ZN not found","meta":{"entityType":"workflow run","entityId":"wrun_01KWEF3VV779FRP13S88W078ZN"}}

The run is being processed by the same deployment's POST /.well-known/workflow/v1/flow handler at the time its stream write is rejected as not-found — i.e. the World does not have a run the deployment itself just started.

Secondary symptom (cascade)

Pre-existing runs also became unresolvable, so delivering to an already-active session throws WorkflowRunNotFoundError, and the fallback new run then cannot claim its hook token:

WorkflowRunNotFoundError: Workflow run "wrun_01K...redacted..." not found
HookConflictError: Hook token "<redacted>" is already in use (run "wrun_01K...redacted...")

Net effect: an inbound event can neither resume the existing run nor start a new one.

Timeline (UTC)
Time (UTC) Event
08:31:05 Last successful run wrun_01KWECV85FHE95SHK6MCSXX4CN
08:57:34 New production deployment created
09:10:18 First failing run wrun_01KWEF321VKHFXQ876FZE8RG2A
~10:04 Fresh deploy from identical code — also fails
~10:08 Rolled production alias back to the 07:43-UTC deployment — service restored
What we ruled out
  • No application code change between the working and failing deployments.
  • Identical dependency pins in the uploaded package.json / bun.lock of the failing deployment (@workflow/core@5.0.0-beta.26, @workflow/world-vercel@5.0.0-beta.22).
  • No project env-var changes (all last modified 9 days prior).
  • Not a serialization/durable-format-version change (format version unchanged).

The only differentiator between working and failing runs is which deployment created them: deployments created before ~08:57 UTC register/resolve runs in the World fine; deployments created after do not.

Expected

A run created by a freshly-deployed function should be registered in the Workflow World so subsequent step/stream writes for that run resolve, regardless of deployment age.

Actual

Runs created by post-08:57-UTC deployments are absent from the World; every step-stream PUT …/runs/<run>/stream/<strm> returns 404 not-found, failing the run.

Questions
  1. Is there a per-deployment Workflow World provisioning/binding step that can fail for a new deployment (leaving @workflow/world-vercel unable to persist runs that deployment creates)?
  2. How can a new deployment be created safely without re-triggering this? (We are currently pinned to a 2-hour-old deployment.)
  3. Is there a supported way to release/expire orphaned hook tokens whose owning runs are no longer resolvable?
  4. Was there a Workflows platform change/incident in fra1 around 2026-07-01 08:57 UTC?
Additional context

Private identifiers (team/project/deployment IDs, affected hook tokens) are in Vercel support ticket 01O1qb7pAz2Gs2tP.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No repository file or test is identified. Start by reviewing the deployment timeline, the failing stream-write examples, and support ticket 01O1qb7pAz2Gs2tP; completion would require a confirmed cause and resolution for runs created by new deployments, including verification that those runs and hook tokens resolve correctly.

Written by the indexing model from the issue text.

Assessment

Tech stack
bun, typescript
Domain
backend, distributed-systems
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.