`SyntaxError: Unexpected end of JSON input` in unknown step name "?" when running on Vercel

I'm noticing a new issue that happens when workflows running on Vercel get called back via a webhook.

Instead of whatever step they were supposed to call, it seems a mysterious step "?" starts instead.

Whether this is a bug in the observability tools or Vercel's Workflows world, I do not know.
However the workflows (which previously completed a few days ago) inevitably fail at this ? step, after an hour where it gets retried 3-6 times (why does vercel retry steps more than the documented 3? We did not configure any custom retry amounts, I verified this)

The error is our only clue to what is causing this problem:

"{\"message\":\"Step \\\"__builtin_response_json\\\" failed after max retries: SyntaxError: Unexpected end of JSON input\",\"stack\":\"SyntaxError: Unexpected end of JSON input\\n    at JSON.parse (<anonymous>)\\n    at parseJSONFromBytes (node:internal/deps/undici/undici:5738:19)\\n    at successSteps (node:internal/deps/undici/undici:5719:27)\\n    at fullyReadBody (node:internal/deps/undici/undici:4609:9)\\n    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)\\n    at async consumeBody (node:internal/deps/undici/undici:5728:7)\\n    at async /var/task/.next/server/chunks/node_modules_d9e94f27._.js:131:1787\\n    at async /var/task/.next/server/chunks/node_modules_d9e94f27._.js:31:45658\\n    at async /var/task/.next/server/chunks/node_modules_d9e94f27._.js:31:45162\\n    at async /var/task/.next/server/chunks/node_modules_d9e94f27._.js:131:302\"}"

Moreover, the observability tools clearly can't load the events for this ? step once it fails:

Here is the code:

async function runFalJobWithWebhook<Id extends EndpointType>(
  endpoint: Id,
  {
    input,
    priority = "normal",
    onEnqueue,
    onQueueUpdate,
  }: {
    input: InputType<Id>
    priority?: QueuePriority
    onEnqueue?: (requestId: string) => Promise<void>
    onQueueUpdate?: (status: QueueStatus) => Promise<void>
  }
): Promise<Result<OutputType<Id>>> {
  "use workflow"
  const workflowCtx = getWorkflowMetadata()
  const parentId = workflowCtx.workflowRunId || null
  const webhook = createWebhook()
  const job = await queueFalJobStep(endpoint, {
    input,
    webhookUrl: webhook.url,
    parentId,
    priority,
  })
  const initialResult = await updateFalJobStep(endpoint, job.external_job_id!)
  if (initialResult) {
    return initialResult as Result<OutputType<Id>>
  }
  for await (const request of webhook) {
    if (request) {
      const falWebhookObject = await request.json()
      const falId: string = falWebhookObject.request_id
      if (falId !== job.external_job_id) {
        console.error(
          `Received webhook for unknown fal ID ${falId}, expected ${job.external_job_id}`
        )
      }
    }
    const result = await updateFalJobStep(endpoint, job.external_job_id!)
    if (result) {
      await falCompletedMarkerStep(endpoint, job.external_job_id!)
      return result as Result<OutputType<Id>>
    }
  }
  throw new FatalError("Workflow terminated without receiving job result")
}

Version:

"workflow": "^4.0.1-beta.19"

EDIT: I can confirm this is a Webhook related issue. Polling with sleep() completely sidesteps it. Is this an issue with request.json()? It doesn't explain the broken events display or why the ? step takes so long (look at the timeline!) or even why it's called "?".

This would imply fal.ai sending us webhook requests with... broken streams? Cutting off midway?

Start with runFalJobWithWebhook and the webhook request.json() path, using workflow 4.0.1-beta.19 and comparing webhook callbacks with the polling-and-sleep path. Reproduce the fal.ai callback failure and trace the response handling and workflow event recording; done means the intended step runs, webhook results complete without retries caused by truncated JSON, and observability no longer shows an unnamed '?' step.