HarperFast / HarperFast/harper

unhandledRejection duplicates the deploy_component 422 when the ingest write transaction times out

Open
#2,056 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
89
Forks
10
Avg merge
2d 2h
Merged PRs (30d)
205

Description

## Summary

When a `deploy_component` write transaction is aborted by the long-transaction monitor
(`transactionOpenTooLongError`, `resources/DatabaseTransaction.ts`), the same rejection that
correctly surfaces as the operation's 422 response *also* escapes as an `unhandledRejection`.
Both of Harper's own `unhandledRejection` listeners (`bin/run.ts` and
`server/threads/threadServer.js`) log it, so it appears twice in the log with two different
formats but is a single underlying event.

## Reproduction

1. `HARPER_TEST_LARGE_DEPLOY_MB=600 npm run test:integration -- integrationTests/deploy/deploy-large-payload.test.ts`
with `storage.maxTransactionOpenTime` left low enough (or disk slow enough) that the
payload-ingest write transaction times out mid-stream.
2. The request correctly resolves 422 with `transactionOpenTooLongError`'s message.
3. Immediately after, the log also shows:
```
[error]: Unhandled promise rejection: Promise Promise { ServerError: Transaction was aborted... } reason: Error: Transaction was aborted...
[error]: unhandledRejection Error: Transaction was aborted after exceeding the maximum open-transaction time...
```
with the identical stack trace as the properly-handled 422:
```
at transactionOpenTooLongError (resources/DatabaseTransaction.js:180:12)
at DatabaseTransaction.addWrite (resources/DatabaseTransaction.js:354:19)
at proceed (resources/Table.js:2767:36)
at TableResource._writeUpdate (resources/Table.js:2769:64)
at TableResource.update (resources/Table.js:1719:45)
at TableResource.put (resources/Table.js:2008:49)
at put.transactional.hasContent (resources/Resource.js:145:28)
at runAction (resources/Resource.js:725:32)
at resources/Resource.js:785:24
at when (utility/when.js:10:12)
```

## What's confirmed vs. still open

- Confirmed: `components/operations.js`'s `deployComponent` catch block does `await
recorder.finish('failed', err)` wrapped in its own try/catch (`Failed to record deployment
failure row`), so *that* specific write's rejection is handled.
- Confirmed: the outer `deployComponent` rejection is caught and logged once by
`callOperationFunctionAsAwait` (`utility/OperationFunctionCaller.ts`) and correctly produces
the 422.
- Not yet isolated: which specific promise is the orphaned one. Candidates investigated and
not yet ruled in or out: a second `table.put()` triggered by `DeploymentRecorder`'s
coalesced-flush path (`scheduleFlush()`/`appendEvent()`, `components/deploymentRecorder.ts`)
racing with `finish()`'s own drain loop; or something in `Table.ts`'s write/retry chaining
(`addWrite`/`save`/`proceed`) that creates more than one promise for the same logical write
when the owning transaction is poisoned mid-flight.

## Why filed separately

harper#large-deploy-txn-abort (root-causing the CI flake this was found alongside) needed a
timely fix for the open-transaction abort itself; tracking down this specific double-promise
requires deeper archaeology in `Table.ts`'s generic write-retry machinery, which is shared by
every write path, not specific to deploy. Isolating it safely deserves its own scoped
investigation rather than a same-PR guess.

## Impact

Cosmetic/log-noise today (the operation still returns the correct error to the caller), but an
unhandled rejection is one Node major version away from being a fatal crash
(`--unhandled-rejections=strict`), and the log duplication makes on-call triage of a real
open-transaction incident noisier than necessary.

Contributor guide

Open the contributing guide

Research direction

Run the listed large-payload integration reproduction, then trace the rejection through resources/DatabaseTransaction.ts, Table.ts, components/deploymentRecorder.ts, and utility/OperationFunctionCaller.ts. Compare the coalesced-flush path with finish() and the addWrite/save/proceed retry chain. Done means identifying and safely handling the orphaned promise, preserving the 422 response, and preventing duplicate unhandled-rejection logs.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, node.js, typescript
Domain
backend, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.