payloadcms / payloadcms/payload

"TypeError: Cannot use 'in' operator to search for '_rels' in undefined" when running parallel jobs with vercel postgres adapter

Open
#13,635 8 comments 5 reactions 1 assignee View on GitHub

@AlessioGr is already working on this.

Since Nov 18, 2025.

stale
Dominant language
TypeScript
Stars
44.8k
Forks
4.2k
Avg merge
2d 21h
Merged PRs (30d)
53

Description

Describe the Bug

When queueing more than 10 jobs in parallel, a TypeError: Cannot use 'in' operator to search for '_rels' in undefined is thrown during the execution of the transform function within upsertRow.

This issue appears to be specific to the Vercel Postgres adapter and doesn't occur with fewer than 10 parallel jobs. The error does not occur with local database (pg.Pool) nor with @payloadcms/db-postgres.

Error Details

  • Error: TypeError: Cannot use 'in' operator to search for '_rels' in undefined
  • Location: transform function called from upsertRow function
  • Root cause: Drizzle findFirst call returns undefined instead of expected result
  • Trigger condition: Only occurs when >10 jobs are queued in parallel
  • Adapter: Vercel Postgres adapter only

Additional Context

  • This issue does not occur with local database:
    // Use non-vercel postgres for local database
    if (
      !this.forceUseVercelPostgres &&
      connectionString &&
      ['127.0.0.1', 'localhost'].includes(new URL(connectionString).hostname)
    ) {
      client = new pg.Pool(
        this.poolOptions ?? {
          connectionString,
        },
      )
    } else {
      client = this.poolOptions ? new VercelPool(this.poolOptions) : sql
    }
  • The problem seems to be related to connection pooling or concurrency limits
  • Error only manifests under high concurrency conditions (>10 parallel jobs)
  • Might be an upstream issue? Although could not reproduce this example with raw drizzle and @vercel/postgres.
Logs 
...
[09:32:59] ERROR: error checking project services
    input: {}
    errorRaw: {}
    error: "TypeError: Cannot use 'in' operator to search for '_rels' in undefined"
...
[09:32:59] ERROR: Error running task 1. Attempt 1 - max retries reached
    job: {
      "id": 1295,
      "input": {},
      "completedAt": null,
      "totalTried": 1,
      "hasError": true,
      "error": {
        "name": "Error",
        "stack": "Error: TypeError: Cannot use 'in' operator to search for '_rels' in undefined\n    at Object.checkService (/code/payload/packages/payload/src/queues/operations/runJobs/runJob/getRunTaskFunction.ts:166:15)\n    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)\n    at async handler (/code/payload/packages/payload/src/queues/operations/runJobs/index.ts:261:13)\n    at async runJob (/code/payload/packages/payload/src/queues/operations/runJobs/runJob/index.ts:46:5)\n    at async runSingleJob (/code/payload/packages/payload/src/queues/operations/runJobs/index.ts:316:24)\n    at async Promise.all (index 9)\n    at async runJobs (/code/payload/packages/payload/src/queues/operations/runJobs/index.ts:369:21)\n    at async Object.run (/code/payload/packages/payload/src/queues/localAPI.ts:186:12)\n    at async run (/code/payload-vercel-postgres-jobs-error/src/playground.ts:69:3)",
        "message": "TypeError: Cannot use 'in' operator to search for '_rels' in undefined",
        "cancelled": false
      },
      "log": [
        {
          "id": "68b157abc9dd50f11e3bafee",
          "executedAt": "2025-08-29T07:32:46.415Z",
          "completedAt": "2025-08-29T07:32:59.082Z",
          "taskSlug": "checkService",
          "taskID": "1",
          "input": {},
          "output": {},
          "state": "failed",
          "error": {
            "name": "Error",
            "stack": "Error: TypeError: Cannot use 'in' operator to search for '_rels' in undefined\n    at Object.checkService (/code/payload/packages/payload/src/queues/operations/runJobs/runJob/getRunTaskFunction.ts:166:15)\n    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)\n    at async handler (/code/payload/packages/payload/src/queues/operations/runJobs/index.ts:261:13)\n    at async runJob (/code/payload/packages/payload/src/queues/operations/runJobs/runJob/index.ts:46:5)\n    at async runSingleJob (/code/payload/packages/payload/src/queues/operations/runJobs/index.ts:316:24)\n    at async Promise.all (index 9)\n    at async runJobs (/code/payload/packages/payload/src/queues/operations/runJobs/index.ts:369:21)\n    at async Object.run (/code/payload/packages/payload/src/queues/localAPI.ts:186:12)\n    at async run (/code/payload-vercel-postgres-jobs-error/src/playground.ts:69:3)",
            "message": "TypeError: Cannot use 'in' operator to search for '_rels' in undefined",
            "cancelled": false
          },
          "parent": {
            "taskSlug": null,
            "taskID": null
          }
        }
      ],
      "taskSlug": "checkService",
      "queue": "default",
      "waitUntil": null,
      "processing": false,
      "updatedAt": "2025-08-29T07:32:59.082Z",
      "createdAt": "2025-08-29T07:32:45.803Z",
      "taskStatus": {
        "checkService": {
          "1": {
            "complete": false,
            "input": {},
            "output": {},
            "taskSlug": "checkService",
            "totalTried": 1
          }
        }
      }
    }
    taskSlug: "checkService"
    err: {
      "type": "TaskError",
      "message": "TypeError: Cannot use 'in' operator to search for '_rels' in undefined",
      "stack":
          Error: TypeError: Cannot use 'in' operator to search for '_rels' in undefined
              at Object.checkService (/code/payload/packages/payload/src/queues/operations/runJobs/runJob/getRunTaskFunction.ts:166:15)
              at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
              at async handler (/code/payload/packages/payload/src/queues/operations/runJobs/index.ts:261:13)
              at async runJob (/code/payload/packages/payload/src/queues/operations/runJobs/runJob/index.ts:46:5)
              at async runSingleJob (/code/payload/packages/payload/src/queues/operations/runJobs/index.ts:316:24)
              at async Promise.all (index 9)
              at async runJobs (/code/payload/packages/payload/src/queues/operations/runJobs/index.ts:369:21)
              at async Object.run (/code/payload/packages/payload/src/queues/localAPI.ts:186:12)
              at async run (/code/payload-vercel-postgres-jobs-error/src/playground.ts:69:3)
      "args": {
        "type": "Object",
        "message": "TypeError: Cannot use 'in' operator to search for '_rels' in undefined",
        "stack":
            
        "executedAt": "2025-08-29T07:32:46.415Z",
        "input": {},
        "job": {
          "id": 1295,
          "input": {},
          "completedAt": null,
          "totalTried": 1,
          "hasError": true,
          "error": {
            "name": "Error",
            "stack":
                Error: TypeError: Cannot use 'in' operator to search for '_rels' in undefined
                    at Object.checkService (/code/payload/packages/payload/src/queues/operations/runJobs/runJob/getRunTaskFunction.ts:166:15)
                    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
                    at async handler (/code/payload/packages/payload/src/queues/operations/runJobs/index.ts:261:13)
                    at async runJob (/code/payload/packages/payload/src/queues/operations/runJobs/runJob/index.ts:46:5)
                    at async runSingleJob (/code/payload/packages/payload/src/queues/operations/runJobs/index.ts:316:24)
                    at async Promise.all (index 9)
                    at async runJobs (/code/payload/packages/payload/src/queues/operations/runJobs/index.ts:369:21)
                    at async Object.run (/code/payload/packages/payload/src/queues/localAPI.ts:186:12)
                    at async run (/code/payload-vercel-postgres-jobs-error/src/playground.ts:69:3)
            "message": "TypeError: Cannot use 'in' operator to search for '_rels' in undefined",
            "cancelled": false
          },
          "log": [
            {
              "id": "68b157abc9dd50f11e3bafee",
              "executedAt": "2025-08-29T07:32:46.415Z",
              "completedAt": "2025-08-29T07:32:59.082Z",
              "taskSlug": "checkService",
              "taskID": "1",
              "input": {},
              "output": {},
              "state": "failed",
              "error": {
                "name": "Error",
                "stack":
                    Error: TypeError: Cannot use 'in' operator to search for '_rels' in undefined
                        at Object.checkService (/code/payload/packages/payload/src/queues/operations/runJobs/runJob/getRunTaskFunction.ts:166:15)
                        at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
                        at async handler (/code/payload/packages/payload/src/queues/operations/runJobs/index.ts:261:13)
                        at async runJob (/code/payload/packages/payload/src/queues/operations/runJobs/runJob/index.ts:46:5)
                        at async runSingleJob (/code/payload/packages/payload/src/queues/operations/runJobs/index.ts:316:24)
                        at async Promise.all (index 9)
                        at async runJobs (/code/payload/packages/payload/src/queues/operations/runJobs/index.ts:369:21)
                        at async Object.run (/code/payload/packages/payload/src/queues/localAPI.ts:186:12)
                        at async run (/code/payload-vercel-postgres-jobs-error/src/playground.ts:69:3)
                "message": "TypeError: Cannot use 'in' operator to search for '_rels' in undefined",
                "cancelled": false
              },
              "parent": {
                "taskSlug": null,
                "taskID": null
              }
            }
          ],
          "taskSlug": "checkService",
          "queue": "default",
          "waitUntil": null,
          "processing": false,
          "updatedAt": "2025-08-29T07:32:59.082Z",
          "createdAt": "2025-08-29T07:32:45.803Z",
          "taskStatus": {
            "checkService": {
              "1": {
                "complete": false,
                "input": {},
                "output": {},
                "taskSlug": "checkService",
                "totalTried": 1
              }
            }
          }
        },
        "output": {},
        "retriesConfig": {},
        "taskConfig": {
          "slug": "checkService",
          "inputSchema": [],
          "outputSchema": []
        },
        "taskID": "1",
        "taskSlug": "checkService",
        "taskStatus": null,
        "workflowConfig": {
          "slug": "singleTask"
        }
      }
    }
Image
Link to the code that reproduces this issue

https://github.com/lubosmato/payload-jobs-vercel-postgres-error

Reproduction Steps
  1. Clone https://github.com/lubosmato/payload-jobs-vercel-postgres-error
  2. Setup Neon or Supabase on Vercel
  3. Run playground.ts: pnpm tsx --env-file=.env src/playground.ts
  4. Observe the TypeError being thrown
Which area(s) are affected? (Select all that apply)

db-vercel-postgres, area: core

Environment Info
This project is configured to use pnpm because /code/payload-vercel-postgres-jobs-error/package.json has a "packageManager" field

Binaries:
  Node: 22.16.0
  npm: 10.9.2
  Yarn: N/A
  pnpm: 10.15.0
Relevant Packages:
  payload: 3.54.0
  next: 15.4.4
  @payloadcms/email-nodemailer: 3.54.0
  @payloadcms/graphql: 3.54.0
  @payloadcms/next/utilities: 3.54.0
  @payloadcms/payload-cloud: 3.54.0
  @payloadcms/richtext-lexical: 3.54.0
  @payloadcms/translations: 3.54.0
  @payloadcms/ui/shared: 3.54.0
  react: 19.1.0
  react-dom: 19.1.1
Operating System:
  Platform: darwin
  Arch: arm64
  Version: Darwin Kernel Version 24.6.0: Mon Jul 14 11:30:40 PDT 2025; root:xnu-11417.140.69~1/RELEASE_ARM64_T6041
  Available memory (MB): 49152
  Available CPU cores: 14

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.