HarperFast / HarperFast/harper
RecordEncoder: 0x42 prefix-less classic-record decode ambiguity keeps resurfacing — write-side guard or raw-put caller audit
- Dominant language
- JavaScript
- Stars
- 89
- Forks
- 10
- Avg merge
- 2d 6h
- Merged PRs (30d)
- 200
Description
## The ambiguity
The rocksdb decode heuristic in `resources/RecordEncoder.ts` (`decode()`) detects a local-timestamp prefix by first byte `66` (0x42) — but 0x42 is also classic shared-structure **record-id #2**. A **prefix-less** classic record that happens to start with that id is misread as timestamp-prefixed: 8 bytes are stripped and the decode corrupts (observed as `Could not find typed structure N` → `null`). Typed structs live at 0x20–0x3f and never collide, so this only surfaces with classic structures on the rocksdb path.
The heuristic's in-code comment documents the collision and the caller-local escape hatch (`options.noMetadata`), added in b8490dc06 when the **audit store** hit it. It has since been hit a second time — #1549, where agent session rows written via raw `primaryStore.put` produced prefix-less records and the whole agent died to "Unknown session"; the fix switched to Resource-level `.put` so rows carry the versioned prefix.
Two bites, two caller-local fixes. Nothing prevents the next raw-put caller from re-tripping it — this issue tracks a structural answer.
## Call sites to triage (non-test, on main)
Version-less raw `primaryStore.put(id, record)` — prefix-less writes, the exposed shape:
- `agent/session.ts` (×6) — fixed when #1549 lands
- `resources/analytics/write.ts:1161`
- `resources/models/analyticsTable.ts:162`
Version-carrying raw puts (`ResourceBridge.ts:161`, `Table.ts:868`, `Table.ts:958`) stage version metadata and should decode unambiguously, but the audit should confirm.
## Options
1. **Write-side guard** — a store that carries per-record version metadata refuses (or warns loudly on) a prefix-less write. A raw put with no staged version to a versioned store is arguably always a bug; this turns silent corruption into an immediate error at the write site.
2. **Call-site audit + lint** — triage the sites above, then a lint rule or wrapper so raw `primaryStore.put` against versioned stores can't be reintroduced casually.
3. **Decode-side sanity check** — before committing to the 8-byte strip, check the implied float64 is a plausible ms-epoch timestamp. Narrows the collision window; doesn't eliminate it.
(1) is the strongest: it fails at the write, where the bug is, instead of at some later read.
Credit: the second diagnosis and the decode-path analysis are from #1549's writeup.
🤖 Posted by [Claude](https://claude.com/claude-code) on behalf of @heskew
Contributor guide
Assessment
This issue has not been assessed yet.