HarperFast / HarperFast/harper
5.1 GA readiness: models/vector subsystem audit findings
- Dominant language
- JavaScript
- Stars
- 89
- Forks
- 10
- Avg merge
- 2d 6h
- Merged PRs (30d)
- 200
Description
A multipass deep review (4 domains × 2 independent runs) plus an API-surface audit of the merged models/AI subsystem surfaced the following. Consolidated as one issue to keep noise down; fixes land as two batch PRs, and the design items need discussion before 5.1 hardens the API.
## Gating for non-beta 5.1 — PR 1: HNSW/vector data integrity (merged: #1234)
Index corruption persists in stored graphs, so these can't be patched after the fact:
- [x] Non-finite vectors poison ranking — no NaN/Infinity guard at index time (the `embedHook.ts:131` comment claims one exists); `bisectInsert(arr, NaN)` returns 0, so a poisoned node becomes rank 1 of every search that reaches it. (`HierarchicalNavigableSmallWorld.ts:100-109`)
- [x] Deleting the entry-point node can blank vector search — the replacement scan runs without the transaction and without excluding the node being deleted; the next insert orphans the surviving graph. (`HierarchicalNavigableSmallWorld.ts:421-447`)
- [x] Update-cleanup sweep erodes reverse edges — removes a neighbor's reverse edge at all levels `0..l` instead of only the level where the old connection existed. With `@embed`, vector updates are the common write path; records silently vanish from search over time. (`HierarchicalNavigableSmallWorld.ts:451-476`)
- [x] Backfill resume is non-idempotent for HNSW — checkpoint resume re-feeds records with no `existingVector`, so existing nodes get new random levels and overwritten connection lists. (`databases.ts:1303-1326`)
- [x] Structural option change resumes from the old checkpoint — interrupted backfill + option change yields one index built under two configs. (`databases.ts:1187,1290-1299`)
- [x] Small: deletes leak the `safeKey→nodeId` mapping; `le` threshold uses strict `<` (and a threshold of `0` is skipped as falsy).
## Gating for non-beta 5.1 — PR 2: models backend/analytics correctness (in review: #1236)
- [ ] `hdb_model_calls` declares nine `indexed: true` attributes that writes never maintain — attribute-filtered billing/usage queries silently return zero rows. (`analyticsTable.ts:57-73,152`)
- [ ] Bedrock can't invoke inference-profile model IDs (`us.anthropic.claude-…` → family `'us'` → throw); `amazon.nova-*` gets a malformed Titan-shaped body. (`bedrock/index.ts:291-299`)
- [ ] No abort gate at tool-dispatch entry — side-effecting handlers still start after the caller has aborted. (`agentLoop.ts:331-337,358-444`)
- [ ] OpenAI `max_tokens` rejected by current reasoning models — needs `max_completion_tokens` for native OpenAI without breaking compat shims. (`openai/index.ts:311-318`)
- [ ] Unbounded memory against a hostile/compromised upstream — non-streaming `res.json()` has no byte cap; streaming tool-call accumulator maps are capped per-entry but unbounded in count. (`backendHelpers.ts:56-62`, all three streaming parsers)
## Decisions needed before the API hardens (discussion)
- `Message.content` is `string`-only — multimodal content parts are a breaking change to the most central type after GA. Decide the shape now even if backends ship text-only (#765 covers the embed side).
- `responseFormat` silently ignored by anthropic/bedrock — proposal: a `jsonSchema` capability bit + fail loud.
- Error classes not exported — callers must string-match `err.name`.
- Config posture inverted — a cosmetic field typo in `models:` is boot-fatal while a backend-name typo / unset env var silently skips into a generic runtime error.
- Bedrock per-family `InvokeModel` bodies vs the Converse API.
## Follow-ups filed
- Search-side automatic embedding (query text on the search descriptor, embedded via the attribute's `@embed` model): #1277 — raised by @dawsontoth on the docs PR (HarperFast/documentation#523).
## Not gating (for visibility)
Agent-loop lifecycle (consumer disconnect mid-tool-round pays for an extra LLM round; abort mid-round leaves a dangling assistant `toolCalls` turn breaking stored-conversation replay), `composeSignal` timer/listener accumulation, Ollama error bodies never drained, `checkSymmetry` full-cost inside every write transaction, embedding-response `index` half-trust, config ergonomics (quoted-number timeouts; `${VAR}` only intercepted on `apiKey`; kind-level key typos). Gemini/vLLM are adequately covered via the openai backend's `baseUrl`; the `/v1/*` gateway work continues in #631 with #514's `openaiStream()`.
---
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Contributor guide
Assessment
This issue has not been assessed yet.