HarperFast / HarperFast/harper
Migration canonical-structures seed (#1455) is discarded by the typed-table runtime (format mismatch) -> workers mint from empty
- Dominant language
- JavaScript
- Stars
- 89
- Forks
- 10
- Avg merge
- 2d 6h
- Merged PRs (30d)
- 200
Description
## Summary
`#1455` persists the migration's canonical classic structures as a **bare named array** at `[Symbol.for('structures'), table]` (`bin/copyDb.ts`). But for a **typed table**, the runtime stores/reads the structures as a `Map{named, typed}` — so the runtime's `RecordEncoder.getStructures` does not recognize the bare array and the typed-structure machinery effectively discards it. Result: at runtime start the durable named dictionary is **empty**, and every v5 worker mints the classic structures **from scratch**. #1455's live-fork prevention is therefore a **no-op for typed tables** ("migration tail is 0" only reflects that migrated records self-describe inline).
## Evidence
On every node of a `4.7.34 → 5.1.14` in-place upgrade, the *first* `saveStructures` for the table reads `oldNamedLen=0` (durable empty) despite #1455 having run and the persist key matching the runtime read key (`events/`). A durable dump confirms the runtime location holds a `Map{named,typed}` (worker-rebuilt), and #1455's bare array is not what the runtime reads.
## Impact
Workers minting from an empty durable concurrently is the precondition for the structure-id divergence (#1506 / the replication-apply issue). Seeding one canonical order at startup would largely prevent the divergence rather than relying on recovery.
## Proposed fix
Persist the canonical seed in the **runtime's format** — merge the canonical `named` into the durable `{named, typed}` representation (preserving typed), or seed via the runtime's own structure-persist path — so every worker adopts one canonical order on startup. Note #1455 deliberately drops v4 typed structs; that interacts with this and should be revisited for typed tables.
Refs #1455, #1506, #1453
— filed by KrAIs (Claude) on Kris's behalf
Contributor guide
Research direction
Start with bin/copyDb.ts and the runtime's RecordEncoder.getStructures and saveStructures paths to compare the migration seed format with the typed-table representation. Verify the fix against a 4.7.34 to 5.1.14 upgrade and confirm that the durable named structures are seeded in canonical order while typed structures are preserved and workers do not rebuild from an empty dictionary.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript, node.js
- Domain
- backend, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100