HarperFast / HarperFast/rocksdb-js

Cross-thread lock-release and parked-commit wakes use unref'd threadsafe functions — a waiting thread's event loop can drain and exit before the wake arrives

Open
#797 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
21
Forks
2
Avg merge
2d 9h
Merged PRs (30d)
36

Description

### Problem

Two cross-thread wake paths deliver their completion through threadsafe functions that are `napi_unref_threadsafe_function`'d at creation:

1. **Lock-release wakes** — `DBDescriptor::lockEnqueueCallback` (db_descriptor.cpp, `NAPI_STATUS_THROWS_VOID(::napi_unref_threadsafe_function(env, threadsafeCallback))` right after creation): the `onUnlocked` callback a waiter enqueues via `Store.tryLock(key, onUnlocked)` / `withLock()`.
2. **Parked-commit retry wakes** — the coordinated-retry `RETRY_NOW` TSFN in `Transaction` commit completion (transaction.cpp, `::napi_unref_threadsafe_function(env, tsfn)` after `napi_create_threadsafe_function`): a commit that hit `IsBusy` parks on the conflicting holder's VT-slot lock and resolves when the holder releases.

In both cases the waiting thread has real pending work — a promise that will be resolved by another thread's action — but contributes **nothing to its own event loop's liveness**. If the waiter has no other ref'd handle at that moment, its loop drains and the thread exits cleanly before the wake arrives. The wake then fires into a dead env (or is dropped).

This is the root cause of HarperFast/harper#2312: a Harper worker thread parked on a peer worker's lock during concurrent multi-worker startup (a pre-ready worker owns no ref'd handles) exits with code 0 before its ready handshake, aborting the whole node's boot — currently the top non-Windows CI failure on harper main. Post-ready workers survive the identical parking only because unrelated ref'd handles (parentPort, listeners) happen to hold their loops.

Compare `DBDescriptor::registerCommitCompletion`, which gets this right: the commit-completion TSFN is ref'd while a commit is pending and unref'd when the pending count returns to zero.

### Expected

A pending cross-thread wake (enqueued lock callback, parked commit) should keep the waiting thread's event loop alive until delivered — ref the TSFN while the wait is outstanding and unref on delivery/cancel, mirroring the pending-commit-completion pattern. Idle registrations (no waiter outstanding) should stay unref'd so threads can still exit normally.

harper is landing a defense-in-depth fix (holding a ref through its workers' pre-ready window), but the library-level contract — "an awaited rocksdb-js operation keeps the loop alive" — belongs here.

Contributor guide

Open the contributing guide

Research direction

Read DBDescriptor::lockEnqueueCallback in db_descriptor.cpp and the coordinated-retry RETRY_NOW TSFN handling in transaction.cpp, then compare both with DBDescriptor::registerCommitCompletion. Trace how each pending wake is created, delivered, and cancelled; done means outstanding waits keep the waiting event loop alive while idle registrations remain unref'd.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, node.js
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.