HarperFast / HarperFast/harper

Single transaction has no row-count write-batch guard -> worker OOM crash (SIGABRT) on many-small-record bulk writes

Open
#1,705 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
JavaScript
Stars
89
Forks
10
Avg merge
2d 6h
Merged PRs (30d)
200

Description

## Summary

A single transaction has **no application-level write-batch guard on row/object COUNT** — only raw payload bytes matter to intuition, but byte-size alone is the wrong signal. A large-enough single-request write set accumulates unboundedly in the JS heap until commit, and past the heap limit the worker hard-crashes (`SIGABRT`, `FATAL ERROR: Reached heap limit ... JavaScript heap out of memory`) instead of returning a clean "too large" rejection.

## Root cause / driver

The crash driver is **per-transaction row/object COUNT, not raw payload bytes**. A request's writes accumulate entirely in-process (JS-heap-backed encoded buffers) until the handler returns and commits, so the batch grows unboundedly with N; the only ceiling is the configured V8 heap limit.

Confirmed via a controlled sweep (512/1024/2048/4096MB heap caps + uncapped default):
- A **few-large-records** shape (~3-6K rows × 1MiB, same total bytes as the crashing case below) never crashed, even at 1.5× the cap in total bytes — large blobs mostly land in Buffer/external allocations and don't pressure V8 old-space the same way.
- A **many-small-records** shape (48,000 rows × 64KB) reliably crashes at the same cap — many discrete write-op JS structures exhaust V8 old-space fast, independent of total byte count.

This makes it a realistic, non-adversarial risk: **bulk-insert / CSV-import (many small rows in one transaction) is a common legitimate API pattern**, not just an adversarial multi-GB payload. On any capped-container production deployment (e.g. a 2048MB heap cap), roughly 48K small rows in one request is enough to crash the worker.

## What's clean (for scoping — not part of the bug)

Atomicity and index-parity hold correctly across the board: tested 62MB→3GB single-transaction write sets (~30+ attempts, 4 runs) — every success has `durable==N && indexCount==N`; the clean-failure case (N=48000/~3GB → 500 "Failed to write transaction log entries...ERR_ABORTED") had `durable=0 && indexCount=0`. Zero torn commits, zero index-vs-primary divergence. The bug is specifically the *lack of a pre-commit size/count guard*, not a correctness/atomicity defect.

## Repro

Harper main `ece7da476` (v5.1.15). Default heap = Node's own RAM-scaled default (no `--max-old-space-size` set anywhere in Harper; measured 4288MB on a 30GiB box, Node 24). At ~2GB attempted under a 2048MB heap cap with the many-small-records shape: `FATAL ERROR: Reached heap limit ... JavaScript heap out of memory`, `SIGABRT` while allocating inside msgpackr-extract's native encode (full stack captured in the test log). Flaky exactly at the boundary (same size+cap → sometimes clean success / sometimes clean 500+rollback / sometimes hard crash) — consistent with a genuine memory-pressure boundary, not a deterministic logic bug.

Test files (not yet promoted to the permanent suite, lineage QA-470 → QA-493): `integrationTests/qa-scratch/qa470-txn-bytesize.test.ts`, `integrationTests/qa-scratch/qa493-oom-threshold.test.ts`.

## Suggested fix direction

Add a row/op-count guard (or per-record overhead accounting, not just cumulative byte total) to the write-batch commit path, so an oversized single-transaction write set is rejected cleanly (e.g. a 413-style response) well before it can pressure the heap into a hard crash.

— KrAIs 🤖 (exploratory QA, on Kris's behalf)

Contributor guide

Open the contributing guide

Research direction

Start with integrationTests/qa-scratch/qa493-oom-threshold.test.ts and qa470-txn-bytesize.test.ts, then trace the write-batch commit path they exercise. Compare many-small-record and few-large-record runs under the documented heap caps. Done means an oversized single transaction is rejected cleanly before heap exhaustion, while atomicity and index parity remain intact.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, node.js
Domain
databases, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.