vercel / vercel/workflow

Queue callback error: Error [WorkflowAPIError]: workflow step XXX not found

Open
#438 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
2.4k
Forks
365
Avg merge
2d 11h
Merged PRs (30d)
169

Description

Hi.
I'm building workflow to process through many items. The goal is to fetch all items from datasource and then proceed through each to replicate it in same datasource with different language. As of now it's just a simple duplication of item with its content and meta. Each duplication process may be different depending on its workflow state, tough each is about 2-4 API calls to data storage in a single step.
Some about 2 hours (~1k items) after the workflow started, it's is throwing an error of not found step, like :

Queue callback error: Error [WorkflowAPIError]: workflow step step_01KB03ZG4MHPKTSYP45DWESM8A not found
    at eq (.next/server/chunks/354.js:41:41)
    at async tI (.next/server/chunks/354.js:41:6880)
    at async (.next/server/chunks/354.js:45:768)
    at async (.next/server/chunks/354.js:5:8119)
    at async (.next/server/chunks/354.js:5:7582)
    at async (.next/server/chunks/354.js:45:316)
    at async default (.next/server/chunks/354.js:41:9464)
    at async L.processMessage (.next/server/chunks/354.js:40:14995)
    at async L.consume (.next/server/chunks/354.js:40:16202) {
  cause: undefined,
  status: 404,
  code: undefined,
  url: 'https://vercel-workflow.com/api/v1/runs/wrun_01KB03ZE75MGRXH35KDCVBY731/steps/step_01KB03ZG4MHPKTSYP45DWESM8A?remoteRefBehavior=resolve'
}

This is a repeatable step that is invoked with different parameters and was successful many times before.
Additionally the workflow is not breaking at this point, continuously and endlessly trying to fetch the not found step (at least for couple next hours before manually canceled) - so additional question would be how to programmatically stop it or retry for workflow process clarity and observability?
I'm testing that in Preview environment of Vercel project.
Are there any limitations that i'm unaware and crossing the line in terms of memory, static chunks, timeouts etc ? Observability section is not quite useful when having so many separated steps (infinite scroll for load more), while consolidating them into one with substeps leads to Vercel Function Timeout as it takes longer than max of 800s.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the queue callback failure from the reported workflow pattern and inspect the workflow run and step URL shown in the issue. The payload names no repository files or tests, so trace the queue consumer and workflow-step lookup before determining whether the missing step, repeated retries, or runtime limits cause the failure. Done should identify the cause and define the supported retry or cancellation behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
backend, distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.