HarperFast / HarperFast/rocksdb-js

Encryption at rest: expose RocksDB EncryptedEnv with an AES cipher

Open
#644 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
C++
Stars
21
Forks
2
Avg merge
2d 9h
Merged PRs (30d)
36

Description

## Summary

Investigate and implement **encryption at rest** for `@harperfast/rocksdb-js`. Upstream RocksDB provides an `EncryptedEnv` framework but ships only a ROT13 example cipher — there is no production-grade cipher out of the box, and our binding currently exposes none of this.

## Toolchain findings (already confirmed)

- **`EncryptedEnv` is already linkable.** `nm` on the vendored `deps/rocksdb/lib/librocksdb.a` shows `NewEncryptedEnv`, `CTREncryptionProvider`, and the `EncryptedFileSystem` machinery are compiled in. No RocksDB rebuild needed. The only cipher present is `ROT13BlockCipher` (anonymous namespace, unusable).
- **AES is available via Node's bundled OpenSSL** (3.5.4 on Node 24; `node_shared_openssl=false`, so symbols are exported from the node binary, and openssl headers ship in the node-gyp dev headers dir). We can implement an `AESBlockCipher` against the EVP API with **no new third-party dependency** — only an added openssl include dir in `binding.gyp`.

## How the open path works (where this plugs in)

1. JS `open(options)` -> `src/binding/database/database.cpp` (~L1146) parses each property via `getProperty` into the `DBOptions` struct (`src/binding/options/db_options.h`).
2. `src/binding/database/db_descriptor.cpp::open` (L662+) builds `rocksdb::Options dbOptions` and calls `DB::OpenForReadOnly` / `TransactionDB::Open` / `OptimisticTransactionDB::Open`.

`EncryptedEnv` plugs in at step 2 via `dbOptions.env = `. The option plumbing mirrors existing options exactly.

## Biggest gotcha: the transaction-log subsystem bypasses RocksDB's Env

`src/binding/transaction_log/transaction_log_file_posix.cpp` writes with raw `::open` / `::pwrite` / `::write` (and `std::ofstream` for flushed-state) — **not** through `rocksdb::Env`/`FileSystem`. These logs carry actual transaction payloads (record data). So `EncryptedEnv` would encrypt SSTs/WAL/MANIFEST/blobs but **leave the transaction logs plaintext** — an incomplete at-rest story. Full coverage requires a parallel encryption layer in the txnlog file code (same cipher + key), touching the most corruption-sensitive part of the codebase (cf. the #573 short-writev history). The txnlog reader (`parse-transaction-log.ts` / native reader) must also decrypt.

## Proposed plan (two phases)

**Phase 1 — spike (RocksDB files only):**
- Implement `AESBlockCipher : rocksdb::BlockCipher` (AES-CTR) using Node's OpenSSL EVP API.
- Wire `NewEncryptedEnv` -> `dbOptions.env` in `db_descriptor.cpp`.
- Plumb a key option: TS type -> `getProperty` -> `DBOptions` (raw key Buffer in options for v1).
- `binding.gyp`: add openssl include dir; validate symbol linkage (esp. **Windows**).
- Benchmark (perf_context exposes encrypt/decrypt timers).

**Phase 2 — full coverage:**
- Encrypt the transaction-log file I/O with the same cipher + key; update the txnlog reader to decrypt.
- Tests: roundtrip, wrong-key-fails, existing-plaintext-DB compatibility/migration, txnlog reader.
- Docs + migration story (cannot transparently convert existing plaintext DBs).

## Open decisions
- **a.** v1 scope: RocksDB files only (documented partial coverage) vs. full coverage incl. txnlogs. Given data sensitivity, lean full — but it roughly doubles effort/risk.
- **b.** Key delivery: raw key Buffer (simplest) vs. key-file path vs. KMS-style callback. Key rotation out of scope for v1.
- **c.** Cipher mode params (AES-128 vs 256; IV/nonce derivation from file offset).

## Effort
Spike: a few days. Full coverage (txnlogs + tests + cross-platform build): meaningfully more. Confidence: high on architecture/wiring; medium on the txnlog work and Windows OpenSSL linkage until the spike proves them out.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.