HarperFast / HarperFast/rocksdb-js
Cross-thread lock-release and parked-commit wakes use unref'd threadsafe functions — a waiting thread's event loop can drain and exit before the wake arrives
- Dominant language
- C++
- Stars
- 21
- Forks
- 2
- Avg merge
- 2d 9h
- Merged PRs (30d)
- 36
Description
### Problem
Two cross-thread wake paths deliver their completion through threadsafe functions that are `napi_unref_threadsafe_function`'d at creation:
1. **Lock-release wakes** — `DBDescriptor::lockEnqueueCallback` (db_descriptor.cpp, `NAPI_STATUS_THROWS_VOID(::napi_unref_threadsafe_function(env, threadsafeCallback))` right after creation): the `onUnlocked` callback a waiter enqueues via `Store.tryLock(key, onUnlocked)` / `withLock()`.
2. **Parked-commit retry wakes** — the coordinated-retry `RETRY_NOW` TSFN in `Transaction` commit completion (transaction.cpp, `::napi_unref_threadsafe_function(env, tsfn)` after `napi_create_threadsafe_function`): a commit that hit `IsBusy` parks on the conflicting holder's VT-slot lock and resolves when the holder releases.
In both cases the waiting thread has real pending work — a promise that will be resolved by another thread's action — but contributes **nothing to its own event loop's liveness**. If the waiter has no other ref'd handle at that moment, its loop drains and the thread exits cleanly before the wake arrives. The wake then fires into a dead env (or is dropped).
This is the root cause of HarperFast/harper#2312: a Harper worker thread parked on a peer worker's lock during concurrent multi-worker startup (a pre-ready worker owns no ref'd handles) exits with code 0 before its ready handshake, aborting the whole node's boot — currently the top non-Windows CI failure on harper main. Post-ready workers survive the identical parking only because unrelated ref'd handles (parentPort, listeners) happen to hold their loops.
Compare `DBDescriptor::registerCommitCompletion`, which gets this right: the commit-completion TSFN is ref'd while a commit is pending and unref'd when the pending count returns to zero.
### Expected
A pending cross-thread wake (enqueued lock callback, parked commit) should keep the waiting thread's event loop alive until delivered — ref the TSFN while the wait is outstanding and unref on delivery/cancel, mirroring the pending-commit-completion pattern. Idle registrations (no waiter outstanding) should stay unref'd so threads can still exit normally.
harper is landing a defense-in-depth fix (holding a ref through its workers' pre-ready window), but the library-level contract — "an awaited rocksdb-js operation keeps the loop alive" — belongs here.
Contributor guide
Research direction
Read DBDescriptor::lockEnqueueCallback in db_descriptor.cpp and the coordinated-retry RETRY_NOW TSFN handling in transaction.cpp, then compare both with DBDescriptor::registerCommitCompletion. Trace how each pending wake is created, delivered, and cancelled; done means outstanding waits keep the waiting event loop alive while idle registrations remain unref'd.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, node.js
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 68/100