HarperFast / HarperFast/harper

Revert #2574 (notify-based runner-lock wake) once rocksdb-js with #849 is pinned

Open
#2,576 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
89
Forks
10
Avg merge
2d 6h
Merged PRs (30d)
200

Description

#2574 is a **temporary workaround**, not the design. It takes the derived-index runner lock without an unlock callback and wakes waiting runners through the readiness buffer's `notify()`, a self-notification skip, and a 5 s retry timer — because with the pinned rocksdb-js, a `tryLock(key, callback)` waiter queued by a worker that is later `terminate()`d aborts the process on Node 22 when another thread unlocks ([rocksdb-js#848](https://github.com/HarperFast/rocksdb-js/issues/848)). That is a bug in the primitive, fixed at the root by [rocksdb-js#849](https://github.com/HarperFast/rocksdb-js/pull/849) (a dying env's queued lock callbacks are released from its cleanup hook; the release paths hold `locksMutex` across the tsfn calls).

The lock's own unlock callback is the simpler and more correct wake: one primitive, and a holder that dies releases natively and wakes the waiters immediately, where the workaround waits for a commit or up to 5 s. Once the fix is pinned, revert.

## Checklist

- [ ] rocksdb-js release containing #849 is published
- [ ] `@harperfast/rocksdb-js` pin bumped in harper `main` (and any release line that took #2574)
- [ ] revert #2574 (`git revert`, resolving `resources/derivedIndexRuntime.ts` back to `tryLock(this.#lockKey, retry)` and dropping `#skipNextNotify` / `#lockBackoff` / `LOCK_RETRY_MILLISECONDS`; restore the removed "synchronous unlock callback" test in `derivedIndexRuntime.test.js`)
- [ ] `unitTests/resources/vectorIndexPlane.test.js` passes on **Node 22** with the new pin and the callback path (this is the suite that aborted `main`'s Node 22 CI)
- [ ] `DESIGN.md § Derived-index runtime › Ownership and wake-up`: replace the "Temporarily … tracked revert" paragraph with the callback description and a one-line note that the notify path existed until rocksdb-js ⟨version⟩

Verification that the fix alone suffices: `main`'s runtime unchanged against a #849 binary passes `vectorIndexPlane.test.js` 22/22 on Node 22.23.1.

Refs #2574, #2430, #2489.

Contributor guide

Open the contributing guide

Research direction

First confirm that a published rocksdb-js release contains #849 and inspect resources/derivedIndexRuntime.ts, derivedIndexRuntime.test.js, and DESIGN.md § Derived-index runtime › Ownership and wake-up. Run unitTests/resources/vectorIndexPlane.test.js on Node 22 with the new pin. Done means reverting #2574, restoring the synchronous unlock callback test, updating the design note, and passing the Node 22 suite.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, nodejs
Domain
backend, databases
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
50/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.