HarperFast / HarperFast/harper
delete_transaction_logs_before takes the process down mid-job on Windows
- Dominant language
- JavaScript
- Stars
- 89
- Forks
- 10
- Avg merge
- 2d 6h
- Merged PRs (30d)
- 200
Description
## What happens
On Windows, `delete_transaction_logs_before` appears to take the Harper process down mid-job. The process then restarts itself and recovers by replaying transaction logs, so the symptom from a client is a short window of refused connections rather than an error response.
## Evidence
Observed once, on the first CI run of a new integration anchor for the purge blast radius ([PR #2399](https://github.com/HarperFast/harper/pull/2399)), in [`Integration Tests 5/6 (Windows, Node.js v24)`](https://github.com/HarperFast/harper/actions/runs/33316309320/job/99270621052). Harper 5.2.5, `storage.engine: rocksdb`, `threads.count: 1`.
The suite had just churned two databases past a transaction-log rotation and flushed both, so each log held a sealed, fully-flushed, purge-eligible file — then issued a database-scoped `delete_transaction_logs_before`. From that instance's own `hdb.log`:
```
14:19:00.935Z [main/0] [notify]: Harper successfully started.
14:19:03.913Z [job/2] [notify]: Starting job: 755c74c5-8e36-427d-9c03-57cdecaaec2c
14:19:05.349Z Starting Harper...
14:19:05.958Z Harper was not properly shutdown, replaying transaction logs to synchronize database
```
The test's next two requests failed with `ECONNREFUSED 127.0.0.3:9925` (operations API) and `ECONNREFUSED 127.0.0.3:9926` (REST) — i.e. inside that restart window. Its later arms, which do their own explicit restart, then passed against the recovered instance, including full primary-data verification.
The same suite is green on Linux across a dozen consecutive runs, and the other Windows integration shards in the same CI run passed.
## Likely mechanism
The purge deletes whole `.txnlog` files. Those files are memory-mapped (`TransactionLog` maps the active write file at its full configured `maxFileSize` on POSIX, and keeps `_logBuffers` mappings for older files). Windows refuses to unlink a file with a live mapping, so an unlink that is fine on POSIX can fail or fault there. That places this in the same family as HarperFast/rocksdb-js#808.
Not yet confirmed by a native-side stack — that is the first thing this issue needs.
## Impact
- No data loss was observed: the same run verified 500/500 primary rows value-exact through REST, `search_by_value` and a full primary-store scan after the crash and again after two further clean restarts.
- What is *not* established is durability for a crash landing at a worse point in the purge. That window is explicitly outside what the new anchor proves, and this issue is the reason it is worth proving.
- Operationally, a scheduled purge on Windows would show up as a periodically restarting node with no error returned to the caller.
## Repro
`integrationTests/database/qa816-purge-blast-radius.test.ts` on [PR #2399](https://github.com/HarperFast/harper/pull/2399) reproduces the setup. It is currently **skipped on Windows** pending this issue — the skip is the only thing keeping that anchor green there, and it should be removed once this is fixed:
```ts
const skipSuite = process.env.HARPER_RUNTIME === 'bun' || process.platform === 'win32';
```
## Suggested next steps
1. Reproduce on a Windows runner with a native stack (or `--report-on-fatalerror`) to confirm the fault is in the log-file unlink and not elsewhere in the job.
2. If confirmed, close the mapping before unlinking on Windows (or defer the unlink), in `@harperfast/rocksdb-js`.
3. Unskip the anchor.
Contributor guide
Research direction
Start with integrationTests/database/qa816-purge-blast-radius.test.ts and reproduce the skipped Windows case on a Windows runner, using a native stack or --report-on-fatalerror. Inspect the database-scoped delete_transaction_logs_before path in @harperfast/rocksdb-js and confirm whether mapped transaction-log files cause the failure. Done means the purge no longer takes down Harper on Windows, the anchor can be unskipped, and its verification passes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript, node.js
- Domain
- databases, operating-systems, testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100