HarperFast / HarperFast/fulltext

Implement BM25, Boolean, phrase, prefix, fuzzy, autocomplete, and highlighting queries

Open
#12 0 comments 0 reactions 1 assignee Claimed by @kylebernhardy View on GitHub
area:api area:native area:performance
Dominant language
Rust
Stars
0
Forks
0
Avg merge
3h 53m
Merged PRs (30d)
18

Description

## Outcome

Implement the agreed English full-text query behavior using Tantivy query primitives.

## Acceptance criteria

- Weighted BM25 uses documented defaults and per-field boosts.
- Term-any, term-all, phrase, bounded prefix/autocomplete, bounded fuzzy, and preview fuzzy-prefix are supported.
- Structured candidate filters use a score-neutral boosted `TermSetQuery`.
- Fuzzy exact preference uses an exact scoring branch plus a lower constant fuzzy branch without double-counting BM25.
- Positions and surface terms are configurable capabilities.
- Highlighting/snippets are configurable and disabled by default.
- Expansion, candidate, clause, deadline, and result-memory ceilings are enforced.
- Stable score ordering and pagination behavior are covered by fixtures.

## Native storage scope — September 14, 2026

Storage is native Tantivy MmapDirectory for standalone and Harper use. The query, ranking and capability acceptance criteria above remain unchanged; there is no hosted storage implementation to qualify.

Current design: [Native Tantivy storage and Harper derived indexes](https://github.com/HarperFast/fulltext/blob/codex/native-storage-design/docs/native-storage-integration.md).

Comment generated by kAIle (GPT-6)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.