HarperFast / HarperFast/harper
Computed @indexed attribute silently indexes undefined when its JS resolver isn't yet registered (per-worker readiness race)
- Dominant language
- JavaScript
- Stars
- 89
- Forks
- 10
- Avg merge
- 2d 6h
- Merged PRs (30d)
- 200
Description
### Summary
A JS-callback `@computed @indexed` attribute (registered at runtime via `tables..setComputedAttribute(name, fn)` in `resources.js`) can be written and read **before its resolver is registered**, producing silently-wrong results with only a `warn`. The resolver is per-worker in-memory state (`resources/Table.ts` `userResolvers`), set when `resources.js` runs on that worker — but the table's route starts serving as soon as the schema loads, and Harper load-balances requests across many http workers (default `cpus-1`) via `SO_REUSEPORT`. There is no invariant that a write/read to such a table waits until the computed resolver is live on the serving worker.
This is the root cause of the flaky `integrationTests/apiTests/computed-indexed-properties.test.mjs` REST block (de-flaked at the test level on branch `kris/fix-computed-indexed-flake` (PR forthcoming); this issue tracks the underlying product hardening).
### Observable failure modes
Given `jsTotalPrice: Float @computed @indexed` with the resolver assigned at runtime in `resources.js`:
1. **Write-time index poisoning (permanent).** A `PUT` handled by a worker whose `resources.js` hasn't finished computes the `@indexed` value with a missing resolver → `jsTotalPrice` is indexed as `undefined`, **frozen at write time**. It never self-heals on read (msgpackr/index value is materialized at write), so `?jsTotalPrice=` filters miss the record for its lifetime. Only signal is:
> `Computed attribute "jsTotalPrice" does not have a function assigned to it. Please use setComputedAttribute(...)`
2. **On-read null on a cold worker (transient).** A `GET ...?select(jsTotalPrice)` served by a not-yet-warm worker recomputes with a missing resolver and returns `null`.
3. **Operations API / main thread never has the resolver.** `search_by_value` (and `get_attributes` generally) recompute computed fields on the **main thread**, where `resources.js` (an http-worker concern) never ran, so a JS-computed attribute always resolves to `null` there. This is why the integration test cannot assert `jsTotalPrice` via `search_by_value`.
### Reproduction
Defer the resolver registration to widen the cold window (simulating a loaded worker):
```js
// resources.js
setTimeout(() => {
tables.Product.setComputedAttribute('jsTotalPrice', (r) => r.price + r.price * r.taxRate);
}, 4000);
```
With default threads on a multi-core box, a `PUT /Product/1` immediately after the route is reachable, then `GET /Product/?jsTotalPrice=119`, misses the record **deterministically** (verified 3/3 on a 19-worker machine). Filter (1) is the durable one; even after all workers warm up, the record stays mis-indexed.
### Why it matters
The current contract silently violates the expectation that an `@indexed` attribute is queryable by its computed value. In production this bites when a client writes to a freshly (re)deployed component before every worker has finished loading `resources.js` — the record is permanently absent from the computed index with no error surfaced to the caller. Case (3) means the ops API is a systematically wrong surface for JS-computed attributes.
### Suggested directions (for discussion)
- **Enforce write-readiness:** don't serve writes (or don't index) for a table until its runtime computed resolvers are registered on that worker — e.g. defer/queue the write, or reject with a retriable error, rather than indexing `undefined`.
- **Fail loud, not silent:** treat "indexed computed attribute with no resolver at write time" as an error (or a startup-guard) instead of a `warn` + `undefined`.
- **Main-thread parity (case 3):** either register `setComputedAttribute` resolvers on the main thread too, or have the ops API read the stored index value for `@indexed` computed attributes instead of recomputing with an absent resolver.
- **Consider reindex-on-late-registration** if the resolver arrives after writes (harder; the above are cheaper and address the common case).
### References
- `resources/Table.ts` — `setComputedAttribute` (`userResolvers`), the computed `resolve` path that warns + returns `undefined` when the resolver is missing.
- Test-level de-flake: `integrationTests/apiTests/computed-indexed-properties.test.mjs` (readiness gate that polls until the resolver is live on all workers).
Contributor guide
Research direction
Start in resources/Table.ts by tracing setComputedAttribute, userResolvers, and the missing-resolver path. Reproduce the cold-worker behavior with the delayed registration shown in the issue, then run integrationTests/apiTests/computed-indexed-properties.test.mjs. Done should prevent writes from silently indexing undefined and clarify correct behavior for reads and operations API access when a resolver is not ready.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript, nodejs
- Domain
- backend-api-design, databases
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100