HarperFast / HarperFast/fulltext

Prove native checkpoint publication and the Harper derived-index path

Open
#7 1 comment 0 reactions 1 assignee Claimed by @kylebernhardy View on GitHub
area:native area:performance
Dominant language
Rust
Stars
0
Forks
0
Avg merge
3h 53m
Merged PRs (30d)
18

Description

## Outcome

Demonstrate real Harper source delivery through the shared runtime and Fulltext backend into native Tantivy files, with restart/replay correctness.

## Acceptance criteria

- First expose generic native publish(payload) and committed-payload readback by reusing engine and existing publication machinery; version the ABI and test plain commit interactions.
- Exercise actual Harper record writes -> elected runner -> projection -> backend -> native apply -> payload-bearing commit -> reader reload -> query.
- Reuse valid files across restart and exact-resume from Harper's validated local cursor; missing/corrupt/incompatible files or source-log gaps rebuild.
- Inject source-commit-before-apply, apply-before-commit, commit-before-reload, and ownership-loss failures; assert exact IDs through upsert/delete/recreate.
- Prove real worker handoff/quiescence and cross-worker query visibility, including publisher loss before refresh notification.
- Verify replicated writes, initial copy/catch-up, eviction, source restore, and local rebuild before query readiness in the Harper-owned integration suite.
- Measure standalone native and integrated Harper with matching corpus/workload plus a no-index Harper control; include unrelated-table traffic and multiple indexes.
- Automate the reproducible cross-repo slice in CI against pinned artifacts; manual builds and process-kill tests do not establish release or power-loss coverage.

Harper integration code and source lifecycle live in Harper; this issue coordinates wrapper prerequisites and paired qualification. Hosted byte storage, chunk layouts, root-flush/WAL-only comparison, and transport feasibility are retired.

## Design

[Native Tantivy storage and Harper derived indexes](https://github.com/HarperFast/fulltext/blob/codex/native-storage-design/docs/native-storage-integration.md). Native Tantivy files are the only delivery target for standalone use and Harper. Proposed APIs and integration gates are not claims of completed implementation.

Comment generated by kAIle (GPT-6)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.