HarperFast / HarperFast/harper
read_audit_log silently under-counts on the paginated (limit) path — group-boundary entry consumed but never yielded
- Dominant language
- JavaScript
- Stars
- 89
- Forks
- 10
- Avg merge
- 2d 6h
- Merged PRs (30d)
- 200
Description
## What happens
`read_audit_log` silently **under-counts on the paginated (`limit`) path only**. Measured: 228 of 12,584 entries missing at `limit=7`, and the loss scales inversely with page size (≈1.8% at limit=7, ≈11.9% at limit=1) — consistent with **one entry dropped per page boundary**. The full-table scan (no `limit`) returns the exact count; only pagination loses rows. This is distinct from harper#1855, which claims the *full scan itself* under-counts a hot key under concurrent load — the opposite signature.
## Mechanism (on `origin/main` @ e16d9616)
`dataLayer/harperBridge/ResourceBridge.ts:773-800` (`groupRecordsInHistory`):
```js
if (enqueued) {
yield enqueued;
count++;
if (limit && limit <= count) { enqueued = undefined; break; }
}
enqueued = { ...new group from current entry... };
```
The entry that *closes* a group (the first entry of the next group, a new distinct timestamp) is consumed from `table.getHistory()` to detect the boundary but is only stored into `enqueued` **after** the `limit` break — so when the page fills, the loop breaks and that boundary entry is discarded, never yielded. Because each `read_audit_log` call re-runs the generator fresh from `start`/`end` with no server-side cursor (`ReadAuditLogObject.ts` has no cursor field; the MCP tool schema at `components/mcp/tools/schemas/operations.ts:239` doesn't even expose `limit`), the dropped boundary entry is lost permanently on every page transition — more boundaries (smaller `limit`) → proportionally more loss.
## Repro
`read_audit_log` a table with a known history count, once unpaginated and once with a small `limit`, page through to the end, and compare totals. Unpaginated matches; paginated is short by ≈(number of page boundaries).
## Suggested fix
Carry the boundary entry across the `limit` break (stash it as the first entry of the next page) instead of discarding it, or thread a real resumable cursor so a page boundary doesn't consume-and-drop.
## Existing work checked
harper#1855 (OPEN) — different, contradictory mechanism (full-scan under-count under concurrent write load; its repro says an isolated read always finds the full history). Not a dup. Searched "audit log pagination cursor" / "read_audit_log limit page" — no hits. Note: `read_audit_log` is marked "Deprecated in favor of `read_transaction_log`" (`operationDescriptions.ts:105`) — relevant to severity framing, but it's a compliance-facing read and the loss is silent.
## What this does not prove
Whether `read_transaction_log` (the recommended replacement) shares the paging path; behavior at very large histories where multiple boundaries fall in one page.
---
*From dispatch QA finding qa-wave-2026072123, verified against harper origin/main e16d9616, 2026-08-30.*
Contributor guide
Research direction
Start in dataLayer/harperBridge/ResourceBridge.ts at groupRecordsInHistory and inspect ReadAuditLogObject.ts plus components/mcp/tools/schemas/operations.ts:239. Reproduce the issue by comparing unpaginated and small-limit read_audit_log totals across pages, then verify that paginated results retain every entry and match the full-table count.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- nodejs, typescript
- Domain
- backend, database
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 72/100