HarperFast / HarperFast/rocksdb-js
Encryption at rest: expose RocksDB EncryptedEnv with an AES cipher
- Dominant language
- C++
- Stars
- 21
- Forks
- 2
- Avg merge
- 2d 9h
- Merged PRs (30d)
- 36
Description
## Summary
Investigate and implement **encryption at rest** for `@harperfast/rocksdb-js`. Upstream RocksDB provides an `EncryptedEnv` framework but ships only a ROT13 example cipher — there is no production-grade cipher out of the box, and our binding currently exposes none of this.
## Toolchain findings (already confirmed)
- **`EncryptedEnv` is already linkable.** `nm` on the vendored `deps/rocksdb/lib/librocksdb.a` shows `NewEncryptedEnv`, `CTREncryptionProvider`, and the `EncryptedFileSystem` machinery are compiled in. No RocksDB rebuild needed. The only cipher present is `ROT13BlockCipher` (anonymous namespace, unusable).
- **AES is available via Node's bundled OpenSSL** (3.5.4 on Node 24; `node_shared_openssl=false`, so symbols are exported from the node binary, and openssl headers ship in the node-gyp dev headers dir). We can implement an `AESBlockCipher` against the EVP API with **no new third-party dependency** — only an added openssl include dir in `binding.gyp`.
## How the open path works (where this plugs in)
1. JS `open(options)` -> `src/binding/database/database.cpp` (~L1146) parses each property via `getProperty` into the `DBOptions` struct (`src/binding/options/db_options.h`).
2. `src/binding/database/db_descriptor.cpp::open` (L662+) builds `rocksdb::Options dbOptions` and calls `DB::OpenForReadOnly` / `TransactionDB::Open` / `OptimisticTransactionDB::Open`.
`EncryptedEnv` plugs in at step 2 via `dbOptions.env = `. The option plumbing mirrors existing options exactly.
## Biggest gotcha: the transaction-log subsystem bypasses RocksDB's Env
`src/binding/transaction_log/transaction_log_file_posix.cpp` writes with raw `::open` / `::pwrite` / `::write` (and `std::ofstream` for flushed-state) — **not** through `rocksdb::Env`/`FileSystem`. These logs carry actual transaction payloads (record data). So `EncryptedEnv` would encrypt SSTs/WAL/MANIFEST/blobs but **leave the transaction logs plaintext** — an incomplete at-rest story. Full coverage requires a parallel encryption layer in the txnlog file code (same cipher + key), touching the most corruption-sensitive part of the codebase (cf. the #573 short-writev history). The txnlog reader (`parse-transaction-log.ts` / native reader) must also decrypt.
## Proposed plan (two phases)
**Phase 1 — spike (RocksDB files only):**
- Implement `AESBlockCipher : rocksdb::BlockCipher` (AES-CTR) using Node's OpenSSL EVP API.
- Wire `NewEncryptedEnv` -> `dbOptions.env` in `db_descriptor.cpp`.
- Plumb a key option: TS type -> `getProperty` -> `DBOptions` (raw key Buffer in options for v1).
- `binding.gyp`: add openssl include dir; validate symbol linkage (esp. **Windows**).
- Benchmark (perf_context exposes encrypt/decrypt timers).
**Phase 2 — full coverage:**
- Encrypt the transaction-log file I/O with the same cipher + key; update the txnlog reader to decrypt.
- Tests: roundtrip, wrong-key-fails, existing-plaintext-DB compatibility/migration, txnlog reader.
- Docs + migration story (cannot transparently convert existing plaintext DBs).
## Open decisions
- **a.** v1 scope: RocksDB files only (documented partial coverage) vs. full coverage incl. txnlogs. Given data sensitivity, lean full — but it roughly doubles effort/risk.
- **b.** Key delivery: raw key Buffer (simplest) vs. key-file path vs. KMS-style callback. Key rotation out of scope for v1.
- **c.** Cipher mode params (AES-128 vs 256; IV/nonce derivation from file offset).
## Effort
Spike: a few days. Full coverage (txnlogs + tests + cross-platform build): meaningfully more. Confidence: high on architecture/wiring; medium on the txnlog work and Windows OpenSSL linkage until the spike proves them out.
Contributor guide
Assessment
This issue has not been assessed yet.