HarperFast / HarperFast/harper
Caching: 'Can load cached indexed data' reproducibly fails, fencing event never reaches indexed subscription
- Dominant language
- JavaScript
- Stars
- 89
- Forks
- 10
- Avg merge
- 2d 6h
- Merged PRs (30d)
- 200
Description
`unitTests/resources/caching.test.js` fails reproducibly on `main`, not flakily:
```
Caching
Can load cached indexed data:
AssertionError [ERR_ASSERTION]: the fencing source fill should reach the indexed subscription
at waitFor (unitTests/waitFor.js:27:11)
at async fenceEvents (unitTests/resources/caching.test.js:872:4)
at async Context. (unitTests/resources/caching.test.js:924:4)
```
`fenceEvents()` publishes a scratch id through `IndexedCachingTable` and waits for that id to arrive on a subscription opened at the top of the test, on the premise that publications are delivered in commit order — so seeing the fenced id proves everything committed earlier was also delivered. The event never arrives within the `waitFor` timeout.
## Why this is real, not a flake or a side effect of another change
- Reproduces on a **single-file** run with nothing else loaded: `npx mocha unitTests/resources/caching.test.js`.
- Confirmed **twice** in CI on the same commit: [HarperFast/harper#2403, run 33333128365](https://github.com/HarperFast/harper/actions/runs/33333128365) — both the initial run and a re-run after a separate, unrelated `HNSW greedy routing above layer 0` failure cleared as flaky.
- Not caused by #2403: that PR touches only `unitTests/components/EntryHandler.test.js` and zero production code, and `caching.test.js` is byte-identical between `main` and that branch. `dist` on the branch is therefore identical to `main`'s.
- Bisected to introduction: `git log -S "the fencing source fill should reach the indexed subscription" -- unitTests/resources/caching.test.js` points at 4802d438 ("Prevent indexed cache TTL test flakes", #2184), which is an ancestor of current `main`.
## Why it can hide
`unitTests/resources/` is excluded from the `test:unit:main` npm script, so a contributor running the documented "main" gate locally will not see this — only `test:unit:resources` or a full CI run surfaces it.
## Open question for whoever picks this up
Whether the fencing subscription genuinely isn't receiving the event (an indexed-cache pub/sub regression) or the test's fencing premise no longer holds against current publish-ordering/timing behavior. Haven't traced past the reproduction and bisection above.
Contributor guide
Research direction
Run `npx mocha unitTests/resources/caching.test.js` and inspect `fenceEvents()` around lines 872 and 924 in `unitTests/resources/caching.test.js`, along with `unitTests/waitFor.js`. Trace whether `IndexedCachingTable` delivers the scratch id to the existing subscription or whether the test's commit-order premise no longer holds; done means the resource test reliably passes with the underlying cause addressed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript, node.js
- Domain
- databases, testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100