Improve cold reads of compressed-only lockboxes toward ZIP parity

Cold file reads from compressed-only lockboxes are slower than equivalent ZIP reads. Investigate and optimize the read path toward ZIP parity without relying on warm caches.

Scope

• Lockbox encryption: Encryption::None.
• Lockbox signing: Signing::None.
• Compression: default Zstd (level 3); ZIP comparison uses Deflate, created with zip -6 -X.
• Vault encryption is unaffected: vaults must always remain encrypted.
• This issue records an investigation for later implementation; no production optimization was made.

Corrected benchmark results

Luna ran a release-mode Rust probe using file-backed lockboxes and in-process zip 8.6.0 (File + ZipArchive). Both read into a Vec; all output bytes were compared with the source outside the timed sections. Results are medians of 15 repetitions. Each cold measurement starts with a newly opened archive handle; the OS page cache was not flushed. These are application-cold measurements, not cold-disk measurements.

File read	Interactive cold	ReadMostly cold	ReadMostly warm	ZIP read
16 MiB repetitive text	39.703 ms	46.307 ms	27.830 ms	12.459 ms
16 MiB deterministic pseudorandom data	115.082 ms	122.424 ms	30.901 ms	26.559 ms
One 1 KiB file in a 2,000-file archive	4.921 ms	4.798 ms	0.002 ms	0.031 ms

Archive opening is excluded above and measured separately:

Archive	Lockbox open (Interactive / ReadMostly)	ZIP open	Lockbox size	ZIP size
16 MiB text	0.178 / 0.177 ms	0.065 ms	10,560 B	57,780 B
16 MiB pseudorandom	0.198 / 0.198 ms	0.067 ms	16,786,752 B	16,779,902 B
2,000 x 1 KiB	2.249 / 2.226 ms	14.270 ms	2,181,440 B	2,299,030 B

The small-file measurement reads just one file, not the entire tree. ZIP opening includes central-directory parsing, so open-plus-first-read favors lockbox in that particular scenario. The incompressible lockbox falls back to stored data despite compression being enabled. The text sample is highly repetitive and is not representative of all text workloads.

ReadMostly's lazy, bounded 64 MiB decoded-frame cache is separate from the decoded-page cache. It dramatically improves the small-file warm result but does not establish universal warm-read parity for large files.

An earlier subprocess-based ZIP comparison was discarded: it included process startup, writing an output file, and rereading it. Use only the corrected in-process results above.

Read-path findings

These are code-level optimization candidates, not profiler-attributed percentages:

1. Deep copies of whole decoded pages. rust/revault_lockbox_api/src/storage/page_cache.rs, PageCache::read_page, clones the decoded page on cache hits and to populate the cache on misses. PagePayload::Normal owns a Vec, so these are payload copies rather than cheap shared references. Reading one object can copy other objects on the same page.
2. Multiple intermediate buffers in the plaintext path. file_format/page.rs, decode_page_with_format, verifies the page checksum, copies the body and object stream, and allocates individual object payloads. lockbox/files.rs, read_file_chunk_compression_frame, then assembles frame segments, verifies the frame digest, decompresses, and copies the requested slice. read_file_range and LockboxFileReader add further copies. ReadMostly can also copy decompressed data into its cache.
3. Repeated chunk-list work. lockbox/files.rs, read_file_range, clones and sorts the file's complete chunk list on each call. lockbox/file_handles.rs, LockboxFileReader::read_internal, calls it for each 2 MiB logical window. This creates avoidable work as the number of chunks grows.
4. Read amplification. A small request processes its containing page and full compression frame. Normal small-file packing targets 4 KiB; BulkImport targets 2 MiB. Large-file frames and seekable-reader windows are 2 MiB. Data pages can contain multiple objects.
5. Repeated reads and decoder setup. A page-cache miss reads the header and then rereads the header with the body, using mutex/seek/read operations. Zstd decoding constructs a fresh FrameDecoder for each decode. Measure these costs before prioritizing them.

Proposed work

1. Add a durable repository benchmark with deterministic fixture generation, comparable in-process ZIP reads, byte verification, and separate open/first-read/warm timings. The exploratory harness is currently temporary, not a committed regression benchmark.
2. Profile allocations, bytes copied, checksumming, decompression, and I/O on the compressed-only path to attribute the gap.
3. Eliminate deep page clones and unnecessary intermediate copies, using borrowed/shared immutable decoded data or transferring buffer ownership as appropriate. Preserve payload lifetimes, mutation isolation, and existing integrity checks.
4. Give the seekable reader a persistent chunk cursor/index; avoid repeated cloning, sorting, and scanning of the entire chunk list.
5. Benchmark the Zstd decoder independently and evaluate buffer/decoder reuse after removing surrounding overhead. Keep the implementation and dependencies pure Rust.
6. Measure small-file/page amplification separately, including archives created with BulkImport. A format/layout change, if needed, should be assessed separately from compatible read-path changes.

Validation and acceptance

• Demonstrate progress toward ZIP parity for application-cold reads, with encryption and signing disabled throughout.
• Cover representative text, binary/incompressible content, packed small files, larger files, sequential reads, and range/seek reads.
• Report archive size, open latency, first-read latency/throughput, warm-read behavior, and memory use; distinguish application-cold from OS-cold.
• Alternate/randomize benchmark ordering, retain distributions, and use controlled disk-cache experiments before making cold-storage claims. The initial probe used fixed ordering and reported medians only.
• Keep checksum validation, corruption handling, sparse-file behavior, seeking, and byte-for-byte correctness intact.
• Do not count a larger cache as a cold-read optimization.

Luna also ran the core library tests: 262 passed, 5 ignored, 0 failed.

Local exploratory artifacts

Harness: /tmp/coldread/Cargo.toml and /tmp/coldread/src/main.rs.
Fixtures: /home/bsutton/git/revault/target/cold-read-plaintext-20260911.
These are local temporary/ignored artifacts and are not portable repository assets. Preserve or replace them with a committed benchmark when picking up this issue.

Command used for the corrected probe:

TMPDIR=/home/bsutton/git/revault/target/compiler-tmp \
CARGO_TARGET_DIR=/home/bsutton/git/revault/target/creation-options \
cargo run --offline --release --manifest-path /tmp/coldread/Cargo.toml

Start with rust/revault_lockbox_api/src/storage/page_cache.rs, file_format/page.rs, lockbox/files.rs, and lockbox/file_handles.rs to trace page, frame, and chunk-list reads. Run the core library tests and replace the temporary /tmp/coldread harness with a committed benchmark covering the stated fixtures and ZIP comparison. Done means measured application-cold progress toward ZIP parity while preserving integrity, seeking, sparse-file behavior, and byte-for-byte correctness.