HarperFast / HarperFast/harper

Caching: 'Can load cached indexed data' reproducibly fails, fencing event never reaches indexed subscription

Open
#2,404 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
JavaScript
Stars
89
Forks
10
Avg merge
2d 6h
Merged PRs (30d)
200

Description

`unitTests/resources/caching.test.js` fails reproducibly on `main`, not flakily:

```
Caching
Can load cached indexed data:
AssertionError [ERR_ASSERTION]: the fencing source fill should reach the indexed subscription
at waitFor (unitTests/waitFor.js:27:11)
at async fenceEvents (unitTests/resources/caching.test.js:872:4)
at async Context. (unitTests/resources/caching.test.js:924:4)
```

`fenceEvents()` publishes a scratch id through `IndexedCachingTable` and waits for that id to arrive on a subscription opened at the top of the test, on the premise that publications are delivered in commit order — so seeing the fenced id proves everything committed earlier was also delivered. The event never arrives within the `waitFor` timeout.

## Why this is real, not a flake or a side effect of another change

- Reproduces on a **single-file** run with nothing else loaded: `npx mocha unitTests/resources/caching.test.js`.
- Confirmed **twice** in CI on the same commit: [HarperFast/harper#2403, run 33333128365](https://github.com/HarperFast/harper/actions/runs/33333128365) — both the initial run and a re-run after a separate, unrelated `HNSW greedy routing above layer 0` failure cleared as flaky.
- Not caused by #2403: that PR touches only `unitTests/components/EntryHandler.test.js` and zero production code, and `caching.test.js` is byte-identical between `main` and that branch. `dist` on the branch is therefore identical to `main`'s.
- Bisected to introduction: `git log -S "the fencing source fill should reach the indexed subscription" -- unitTests/resources/caching.test.js` points at 4802d438 ("Prevent indexed cache TTL test flakes", #2184), which is an ancestor of current `main`.

## Why it can hide

`unitTests/resources/` is excluded from the `test:unit:main` npm script, so a contributor running the documented "main" gate locally will not see this — only `test:unit:resources` or a full CI run surfaces it.

## Open question for whoever picks this up

Whether the fencing subscription genuinely isn't receiving the event (an indexed-cache pub/sub regression) or the test's fencing premise no longer holds against current publish-ordering/timing behavior. Haven't traced past the reproduction and bisection above.

Contributor guide

Open the contributing guide

Research direction

Run `npx mocha unitTests/resources/caching.test.js` and inspect `fenceEvents()` around lines 872 and 924 in `unitTests/resources/caching.test.js`, along with `unitTests/waitFor.js`. Trace whether `IndexedCachingTable` delivers the scratch id to the existing subscription or whether the test's commit-order premise no longer holds; done means the resource test reliably passes with the underlying cause addressed.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, node.js
Domain
databases, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.