payloadcms / payloadcms/payload

`payload run <script>` intermittently exits 0 without ever evaluating the target module — no error, no output, ~15-20% of invocations under back-to-back CLI use

Open
#17,757 3 comments 0 reactions 1 assignee View on GitHub

@AlessioGr is already working on this.

Since Aug 12, 2026.

Dominant language
TypeScript
Stars
44.8k
Forks
4.2k
Avg merge
2d 21h
Merged PRs (30d)
53

Description

Versions

  • payload: 3.87.1
  • @payloadcms/db-postgres: 3.87.1
  • tsx (nested dependency of payload): 4.22.4
  • Node.js: v22.23.2
  • OS: Linux (Ubuntu 24.04-based), x86_64, 8 cores

Summary

payload run <path-to-ts-file> sometimes exits with code 0 having never evaluated the
target file at all — not even its first top-level statement. No error is printed on
stdout or stderr. This happens intermittently (measured 12/70 ≈ 17% locally, matches
independent reports of 15–20% and 2/10 on GitHub Actions) when payload run (or
payload migrate) is invoked repeatedly, back to back, with no delay between
invocations.

We instrumented the CLI itself (not the target script) with file-based tracing
(fs.appendFileSync, chosen specifically so the trace can't be lost to whatever might
be swallowing stdout/stderr) and captured the exact divergence point across 5 silent
occurrences. All 5 show the identical shape:

bin.js:   tsImport('./dist/bin/index.js', url)  -> resolves fine, always
dist/bin/index.js runBinScript(), script==='run' branch:
  scriptPath resolved                            -> logged, always
  absoluteScriptPath resolved                     -> logged, always
  await import(pathToFileURL(absoluteScriptPath).toString())
                                                   -> called (logged immediately before)
                                                   -> NEVER settles: no success continuation,
                                                      no catch block, no finally block
  [1.8s – 17.1s of nothing]
  process exits, code 0

No uncaughtException, no unhandledRejection, no thrown error anywhere in the
process (verified with global handlers wired at the very top of bin.js, before
anything else runs). The promise from that single import() call simply never settles,
and the process exits cleanly once nothing else is keeping the event loop alive.

Minimal repro

We do not yet have a repro reduced to a bare Payload project (that's the natural next
step for whoever picks this up upstream — our repro is on a ~30-collection production
config). What we have, reduced as far as time allowed:

# any Payload 3.87.1 project with a postgres/mongo adapter
set -a; source .env.local; set +a
for i in $(seq 1 40); do
  npm run <a script that runs `payload run some-file.ts`> > /tmp/o 2>/tmp/e
  c=$?
  # some-file.ts's first top-level statement should write a marker to stderr,
  # flushed, before anything else — including before any guard/throw logic
  grep -q 'MARKER' /tmp/e || echo "run $i: SILENT, exit=$c, stdout=$(wc -c </tmp/o)B stderr=$(wc -c </tmp/e)B"
done

Trigger conditions, as measured:

  • Invocations must be back-to-back with effectively no delay (a for loop with no
    sleep). We have not tested whether a small delay between invocations
    suppresses it (an earlier, less careful local harness that inserted schema-teardown
    time between runs saw 0/27 — see caveats below on why we don't treat that as
    conclusive).
  • NODE_ENV !== production and typescript.autoGenerate not explicitly disabled (see
    next section for why this matters) — though we have not isolated whether autoGenerate
    itself is required to reproduce, only that it is present and firing in every run we
    captured.

A concerning side effect we found while instrumenting, possibly related

Not filed as the main bug, but found investigating this one and worth flagging in the
same report because it may be a contributing factor:

getPayload()'s init() (packages/payload/src/index.ts, compiled to
dist/index.js around line 359 in 3.87.1) does this on every non-production init
unless config.typescript.autoGenerate === false:

if (process.env.NODE_ENV !== 'production' && this.config.typescript.autoGenerate !== false) {
    void this.bin({ args: ['generate:types'], log: false });
}

this.bin() spawns node <path-to-payload>/bin.js generate:types as a **real,****
**unawaited child process (spawn(..., { stdio: 'ignore' }), wrapped in void). This
means:

  1. Every getPayload() call in a dev-mode script silently launches a second, full CLI
    bootstrap (its own bin.jstsImport()dist/bin/index.jsgenerateTypes()
    sequence) as a detached side effect, with no log line indicating it happened (log: false).
  2. Because it's unawaited, a script that calls process.exit() itself right after
    getPayload() finishes its own work (a documented, intentional pattern for scripts
    that hold a DB pool open — see payloadcms/payload#13744 / #13564 for the
    generate:types analogue of the same pool-holds-loop-open problem) does not
    wait for this child. It becomes an orphan.
  3. We measured 3–5 such orphans alive concurrently during a 40-iteration back-to-back
    payload run loop.
  4. Separately, payloadcms/payload#15553 documents that generate:types itself can hang
    indefinitely (never calling process.exit()) under certain plugin/config
    combinations, which would make these orphans long-lived rather than self-clearing.
    We did not confirm #15553's specific hang in our own environment (our orphans did
    eventually exit), but the mechanism — this exact fire-and-forget child — is the same
    one.

We are not claiming this side effect is the direct cause of the silent import().
We could not prove causation in the time available — see FINDINGS.md's explicit
measured-vs-inferred breakdown, in particular the note that the "3 vs 4 orphans"
correlation we observed alongside every silent run is fully explained by simple
bookkeeping (a run that never reaches getPayload() never spawns its own orphan) and
is not independent evidence of resource contention. We flag it because:

  • it's a second, independently-surprising behavior discovered on the same call path
    (bin.js's tsImport()/loader-hook machinery) that this bug lives on,
  • it multiplies the number of concurrent tsx-loader-hook-using processes on the machine
    in exactly the scenario (back-to-back CLI invocations) that triggers the main bug,
    and
  • payloadcms/payload#16734 independently documents that tsImport()'s "scoped"
    registration (exactly what bin.js uses, at the exact two call sites instrumented
    here) has a known gap vs. tsx's normal global registration for propagating resolution
    to nested/dynamic imports — which is the same shape as our second, silently-orphaned
    await import() call.

If maintainers confirm no connection, this half of the report can be split off as its
own issue: "getPayload() silently spawns an unawaited generate:types child on every
dev-mode init; should be logged and/or awaited (or at minimum, made unref()-safe and
loud)."

What we ruled out

  • Not a crash: zero uncaught exceptions, zero unhandled rejections, across 40 traced
    invocations (parent processes and orphan children).
  • Not stdout/stderr buffering hiding a later crash: the gap begins at the import()
    call itself, before the target module can have produced any output on any stream.
  • Not specific to our seed script's logic or guards: the target module's very first
    top-level statement — placed above all guard/throw logic specifically to test this —
    never runs. The bug is upstream of anything in the target file.
  • Not unique to loading any module through the tsx-hooked realm: the CLI's own first
    tsImport() call (loading dist/bin/index.js, plain compiled JS, no transform
    needed) always eventually resolves in every run we captured, healthy or silent, even
    under multi-second delays from machine load. Only the second import — of a .ts file
    requiring tsx's transform — ever fails to settle.
  • Not concurrency-only: our primary reproduction recipe is strictly sequential
    back-to-back invocations (a for loop with no &/parallelism), not concurrent
    processes. (A related but distinct symptom — concurrent payload migrate invocations
    also silently no-oping — is reported separately on our internal tracker; we have not
    established whether it shares this same mechanism.)

Caveat on our own earlier (non-)reproduction

We previously ran ~27 clean local iterations across three configurations before finally
reproducing this. What changed was inserting schema teardown (or not) between
iterations — teardown-between-runs iterations were clean; back-to-back, no-teardown
iterations reproduce at the stated rate. We have not established that teardown
itself is causal (as opposed to simply adding a delay, or changing what work happens
before the next payload run), and flag this explicitly so nobody over-reads it: at a
15–20% true rate, 12 clean serial runs individually has ~14% probability of happening by
chance alone.

Request

We'd appreciate:

  1. Any insight into what could cause a module.register()-based ESM loader hook
    (installed via tsx/esm/api's tsImport(), as bin.js does) to leave a dynamic
    import() promise permanently unsettled with no error surfaced anywhere.
  2. Confirmation of / more context on whether payloadcms/payload#16734's "scoped
    registration gap" for tsImport() could plausibly manifest this way for a second,
    nested import() call issued from inside the already-tsImport()-loaded module,
    rather than from a config-file-relative extensionless import as in that report.
  3. Whether the unawaited generate:types child spawned from getPayload()'s init()
    is intentional, and whether it should be logged, awaited, or otherwise made safe for
    back-to-back invocation.

Happy to provide the full instrumented node_modules/payload diff and the raw trace
log (1395 lines, 40 runs) on request.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.