HarperFast / HarperFast/rocksdb-js

Assess exact replay and retention support for the shared Harper derived-index protocol

Open
#836 0 comments 0 reactions 1 assignee Claimed by @kylebernhardy View on GitHub
Dominant language
C++
Stars
21
Forks
2
Avg merge
2d 9h
Merged PRs (30d)
36

Description

## Outcome

Resolve the exact replay and retention guarantees needed by Harper's shared derived-index protocol
using the supported Harper/storage APIs. This is a protocol investigation, not approval to add a
native cursor table, native storage lease or reservations service to rocksdb-js.

## Acceptance criteria

- Trace current committed-entry positions and resume behavior, including transaction boundaries,
multiple origin logs, log recreation and missing retained history.
- Prove exact coverage and gap detection with local/replicated commits and crash/restart fixtures.
- Keep log encoding internal to its existing owner; fulltext must not parse physical log files or
infer ordering from opaque bytes.
- Identify how Harper protects or validates replay history during rebuild, boot, restore and purge.
Missing history triggers rebuild; it cannot silently advance a derived watermark.
- Document precisely any guarantee unavailable through the supported stack and take that gap back
to the shared Harper protocol design before implementation. No base-library addition is assumed.
- Preserve existing timestamp APIs and behavior; do not conflate approximate time-based resume with
demonstrated exact source progress.
- This issue remains open independently of the retired fulltext native-storage epic.

## References

[Derived-index delivery protocol (DerivedIndexBackend): shared post-commit delivery, watermark/replay, and blob-content contract for HNSW and full-text indexes](https://github.com/HarperFast/harper/issues/2489)
owns the protocol. [Phase 0: prove the Harper-backed Tantivy storage and derived-index path](https://github.com/HarperFast/fulltext/issues/7)
owns the real fulltext storage integration proof.

Comment generated by kAIle (GPT-6)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.