vercel / vercel/workflow

A sleep() that loses a Promise.race can never be closed, and one abandoned timer disables retention and inline deltas for the rest of the run

Open
#3,916 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
2.4k
Forks
365
Avg merge
2d 11h
Merged PRs (30d)
169

Description

Summary

There is no way to dispose a wait. Hooks have hook_created, hook_received and hook_disposed, but waits only have wait_created and wait_completed. So if a workflow races a hook against a sleep as a timeout, and the hook wins, the losing sleep stays open in the journal until its timer elapses. There is nothing the workflow can do about it.

That would just be untidy, except open waits also gate the fast paths. In beta.42, canRetainWorkflowSession and the inline delta path both check for open hooks and waits, and in beta.46 getRetentionDecision still returns false for any open wait (runtime.js, case 'wait'). The turbo optimistic start also never returns once any wait has appeared in the log, since it requires attempt 1 with no incoming step.

What we measured (production, managed world, iad1, core beta.42)

Our human approval gate was written as the docs suggest, hook based, but with a 24 hour timeout added:

const outcome = await Promise.race([
  hook,
  sleep("24h").then(() => ({ kind: "timeout" })),
]);

The user approved 89 seconds into a run. The hook resolved in 13 seconds. The abandoned sleep stayed open for the remaining 62 minutes of the run, and:

  • step boundary re-entry went from 218ms median (43 steps before the gate) to 4,382ms after, a 20x regression that ramped with journal growth and never recovered
  • roughly 19 percent of the total wall clock of that run went to re-entry alone

The pre-gate segment is a clean control: same code, same VM, same run.

Workaround that fixed it for us

We removed the in-run timeout entirely. The gate now parks on the hook alone, and a cron job resumes the hook with a timeout payload if nobody answers within 24 hours. After that change (plus beta.46 so open hooks retain), step boundaries stay at about 108ms for entire runs, including after approvals.

Ask

Either of these would solve it properly:

  1. A way to dispose an abandoned wait, for example sleep returning a handle with a dispose method, or the race pattern being detected.
  2. Retention and the inline delta path ignoring waits whose resolution can no longer affect control flow.

Versions: workflow 5.0.0-beta.42 and 5.0.0-beta.46, @ai-sdk/workflow 1.0.30, Vercel managed world. Full timing methodology available if useful, it all comes from workflow inspect events plus computing step_created minus previous step_completed per boundary.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with runtime.js, especially the getRetentionDecision wait case and the checks for open waits affecting retention and inline deltas. Use the workflow inspect events and the Promise.race timeout scenario to trace the abandoned wait lifecycle. Done means abandoned waits can be disposed or are excluded when they can no longer affect control flow, without regressing retention or step-boundary performance.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
backend, performance
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.