HarperFast / HarperFast/harper

Computed @indexed attribute silently indexes undefined when its JS resolver isn't yet registered (per-worker readiness race)

Open
#1,631 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
89
Forks
10
Avg merge
2d 6h
Merged PRs (30d)
200

Description

### Summary

A JS-callback `@computed @indexed` attribute (registered at runtime via `tables..setComputedAttribute(name, fn)` in `resources.js`) can be written and read **before its resolver is registered**, producing silently-wrong results with only a `warn`. The resolver is per-worker in-memory state (`resources/Table.ts` `userResolvers`), set when `resources.js` runs on that worker — but the table's route starts serving as soon as the schema loads, and Harper load-balances requests across many http workers (default `cpus-1`) via `SO_REUSEPORT`. There is no invariant that a write/read to such a table waits until the computed resolver is live on the serving worker.

This is the root cause of the flaky `integrationTests/apiTests/computed-indexed-properties.test.mjs` REST block (de-flaked at the test level on branch `kris/fix-computed-indexed-flake` (PR forthcoming); this issue tracks the underlying product hardening).

### Observable failure modes

Given `jsTotalPrice: Float @computed @indexed` with the resolver assigned at runtime in `resources.js`:

1. **Write-time index poisoning (permanent).** A `PUT` handled by a worker whose `resources.js` hasn't finished computes the `@indexed` value with a missing resolver → `jsTotalPrice` is indexed as `undefined`, **frozen at write time**. It never self-heals on read (msgpackr/index value is materialized at write), so `?jsTotalPrice=` filters miss the record for its lifetime. Only signal is:
> `Computed attribute "jsTotalPrice" does not have a function assigned to it. Please use setComputedAttribute(...)`

2. **On-read null on a cold worker (transient).** A `GET ...?select(jsTotalPrice)` served by a not-yet-warm worker recomputes with a missing resolver and returns `null`.

3. **Operations API / main thread never has the resolver.** `search_by_value` (and `get_attributes` generally) recompute computed fields on the **main thread**, where `resources.js` (an http-worker concern) never ran, so a JS-computed attribute always resolves to `null` there. This is why the integration test cannot assert `jsTotalPrice` via `search_by_value`.

### Reproduction

Defer the resolver registration to widen the cold window (simulating a loaded worker):

```js
// resources.js
setTimeout(() => {
tables.Product.setComputedAttribute('jsTotalPrice', (r) => r.price + r.price * r.taxRate);
}, 4000);
```

With default threads on a multi-core box, a `PUT /Product/1` immediately after the route is reachable, then `GET /Product/?jsTotalPrice=119`, misses the record **deterministically** (verified 3/3 on a 19-worker machine). Filter (1) is the durable one; even after all workers warm up, the record stays mis-indexed.

### Why it matters

The current contract silently violates the expectation that an `@indexed` attribute is queryable by its computed value. In production this bites when a client writes to a freshly (re)deployed component before every worker has finished loading `resources.js` — the record is permanently absent from the computed index with no error surfaced to the caller. Case (3) means the ops API is a systematically wrong surface for JS-computed attributes.

### Suggested directions (for discussion)

- **Enforce write-readiness:** don't serve writes (or don't index) for a table until its runtime computed resolvers are registered on that worker — e.g. defer/queue the write, or reject with a retriable error, rather than indexing `undefined`.
- **Fail loud, not silent:** treat "indexed computed attribute with no resolver at write time" as an error (or a startup-guard) instead of a `warn` + `undefined`.
- **Main-thread parity (case 3):** either register `setComputedAttribute` resolvers on the main thread too, or have the ops API read the stored index value for `@indexed` computed attributes instead of recomputing with an absent resolver.
- **Consider reindex-on-late-registration** if the resolver arrives after writes (harder; the above are cheaper and address the common case).

### References

- `resources/Table.ts` — `setComputedAttribute` (`userResolvers`), the computed `resolve` path that warns + returns `undefined` when the resolver is missing.
- Test-level de-flake: `integrationTests/apiTests/computed-indexed-properties.test.mjs` (readiness gate that polls until the resolver is live on all workers).

Contributor guide

Open the contributing guide

Research direction

Start in resources/Table.ts by tracing setComputedAttribute, userResolvers, and the missing-resolver path. Reproduce the cold-worker behavior with the delayed registration shown in the issue, then run integrationTests/apiTests/computed-indexed-properties.test.mjs. Done should prevent writes from silently indexing undefined and clarify correct behavior for reads and operations API access when a resolver is not ready.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, nodejs
Domain
backend-api-design, databases
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.