HarperFast / HarperFast/harper

Graceful shutdown doesn't await scope 'close' cleanup — app teardown races process.exit

Open
#1,912 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
89
Forks
10
Avg merge
2d 6h
Merged PRs (30d)
200

Description

## Summary

The supported pattern for application shutdown cleanup is a `'close'` listener on the plugin-API `Scope`:

```js
export function handleApplication(scope) {
const service = startService();
scope.once('close', async () => {
await service.close();
});
}
```

But nothing in the shutdown path waits for that cleanup. Async `'close'` listeners race `process.exit`, and on an idle server the cleanup may not even start before the process exits. This came up as a support question from an app developer; the recommended answer only works best-effort today.

## Mechanics (all on the worker thread, triggered by the `SHUTDOWN` ITC message)

1. **`scope.close()` is fire-and-forget.** `componentLoader.ts:472` registers `onMessageByType(ITC_EVENT_TYPES.SHUTDOWN, () => scope.close())`, and `notifyMessageListeners` (`server/threads/manageThreads.js:610-616`) invokes typed listeners synchronously, dropping returned promises.
2. **Async `'close'` listener promises are dropped.** `Scope.close()` (`components/Scope.ts:181-187`) awaits entry-handler and options teardown, then calls plain `this.emit('close')` — `EventEmitter.emit` does not await listener promises.
3. **Exit is sequenced only on HTTP drain.** `threadServer.js:175-177` does `closeServers().then(() => realExit(0))`, where `realExit` is the original `process.exit`. `closeServers()` resolves when every server's `close()` callback fires (or a 5s per-server cap) — on an idle server that's near-immediate.
4. Because `scope.close()` awaits entry-handler/options teardown *before* emitting `'close'`, on a fast drain the `'close'` event may fire **after** `realExit(0)` has been queued — cleanup never starts.
5. The backstop timers (`threadTerminationTimeout` = 10s worker-side at `manageThreads.js:655-663`, 2× on the main thread before `worker.terminate()` at `manageThreads.js:296-306`) are upper bounds only — nothing ever *waits for* scope cleanup, so finishing under the timeout doesn't help.

Note this affects graceful **restarts** too (dev watch, restart operation) — same `SHUTDOWN` message path via `restartWorkers`.

## Impact

Apps that flush buffers, drain queues, or close connections to external services in a `'close'` listener can be cut off mid-flight (or never run) on shutdown/restart. Typical symptoms: lost telemetry/batches, unclean client disconnects, external resources left open.

## Suggested direction

- Make the worker's exit condition `Promise.allSettled([closeServers(), allScopeCloses])` (bounded by `threadTerminationTimeout`) instead of `closeServers()` alone.
- Have `Scope.close()` await async `'close'` listener results — `emit` can't do this, so either iterate `rawListeners('close')` and await returned promises, or add an explicit registration API (e.g. `scope.onClose(async () => ...)`) and document `'close'` listeners as sync-only.

Docs follow-up (documenting the cleanup pattern + this caveat) tracked separately in the documentation repo.

Contributor guide

Open the contributing guide

Research direction

Start at the SHUTDOWN handler in componentLoader.ts:472 and trace notification in server/threads/manageThreads.js:610-616, Scope.close() in components/Scope.ts:181-187, and exit sequencing in threadServer.js:175-177. Confirm that shutdown and restart wait for asynchronous scope cleanup before process.exit, while respecting the existing termination bounds.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, node.js
Domain
backend
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.