HarperFast / HarperFast/harper

Optionally keep a node out of rotation (or gate component load) until secondary indexes are built

Open
#1,358 0 comments 0 reactions 0 assignees View on GitHub
area:clustering area:search enhancement
Dominant language
JavaScript
Stars
89
Forks
10
Avg merge
2d 6h
Merged PRs (30d)
200

Description

Part of #1354

> **Design discussion** — this presents options and is open for maintainer input; it is not a single prescribed change.

## Problem
A worker begins serving as soon as components load; index backfills (`runIndexing`) run in the background, so a query against a still-building index returns `503 "not indexed yet"` (`resources/search.ts:335-336`). An app that relies on a secondary index immediately at startup — e.g. a bulk import that dedups with `search()` before `create()` — can run against an incomplete index during that window. Harper has no built-in, opt-in way to say "don't take traffic until my indexes are ready."

## Options (both opt-in, default-off)

### A — Node availability gate (preferred where a traffic director fronts the node)
Node out-of-rotation already exists today via the `availability` status → `set_status` → GTM/load-balancer path (`server/status/definitions.ts:13-14`, `utility/hdbTerms.ts:300`). **What's new is auto-driving that availability from index-build state** ("smarter availability"): report **Unavailable** while a declared index is (re)building so the director routes new traffic — reads *and* writes/imports — to ready peers, then flip to **Available** on completion. Fleet-safe: one node out at a time during a rolling rebuild keeps the service up and sends imports to a node whose index is built → no 503/skip. Needs a **quorum guard** (never eject the last/majority simultaneously) and anti-flap. This variant can also re-apply on a *runtime* reindex (self-eject → rebuild → rejoin) under the guard.

### B — Load-time listen gate (opt-in; for deployments with nothing in front, or the simplest form)
Make component load await `Table.indexingOperation` for the dependent table(s) before the worker reports ready / listens (the listen socket already gates on load completion — `server/threads/threadServer.js:166-203`, listen `:190`, `CHILD_STARTED :201`; await hook at `components/componentLoader.ts:251` or `server/loadRootComponents.js:37-41`). **Load-time only** — cannot cover reindexes after the worker is serving.

## Surface
Per-table directive (e.g. `waitForIndexOnLoad`) and/or a component-level readiness flag; a configurable **timeout** (reuse the `handleApplication` timeout pattern, `components/componentLoader.ts:223-265`) so it logs-and-proceeds rather than hanging forever.

## Sharp edges
- `Table.indexingOperation` **resolves even when the backfill failed** — `runIndexing` swallows errors and leaves `isIndexing=true`/`indexingFailed` (`resources/databases.ts:1530-1547,1575-1595`). So awaiting it does not prove completeness; the gate must re-check `isIndexing`/`indexingFailed` and decide (stay-out vs warn-and-serve).
- **Depends on the don't-park fix** (sibling issue): otherwise a transient backfill error parks the index and the gate holds the node out indefinitely.

## Scope / not a silver bullet
Variant B is load-time only and does **not** cover runtime reindexes (added attribute, schema change, crash-recovery after serving) — those still need the typed retryable 503 (sibling issue). This is a **complement**, not a replacement.

## Compatibility
Opt-in, default-off → behavior unchanged when unset (`indexingOperation` stays fire-and-forget today, `resources/databases.ts:1368`). Additive.

---

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.