[Feature] Support primary-key table full-text and hybrid search (read side)
- Dominant language
- Rust
- Stars
- 197
- Forks
- 92
- Avg merge
- 1d 16h
- Merged PRs (30d)
- 91
Description
### Search before asking
- [x] I searched in the [issues](https://github.com/apache/paimon-rust/issues) and found nothing similar.
### Motivation
Paimon (Java) supports full-text search and hybrid (vector + full-text) search on primary-key tables, backed by the shared native `paimon-ftindex-core` engine. The Rust read side already supports primary-key **vector** search end-to-end; the natural next step is to read the primary-key **full-text** and **hybrid** results Java writes, so that Rust / Doris / DataFusion consumers get the same primary-key search capabilities.
### Solution
Mirror the Java PK read path, reusing the already-merged PK-vector machinery and depending on `paimon-ftindex-core` (the same core the Java/Python bindings use), gated behind the `fulltext` feature:
- Read `full-text` index archives via `paimon-ftindex-core` (draft PR #563 lands the reader foundation).
- PK full-text data layer (config options + the shared `index/pk` source-metadata + bucket-state reconciliation) and read path (`PrimaryKeyFullTextScan` / `PrimaryKeyFullTextBucketSearch` / `PrimaryKeyFullTextRead`, FAST mode), materializing rows by physical position — mirroring Java.
- Hybrid search on primary-key tables: fuse vector + full-text routes in physical-position space via a `PrimaryKeySearchRanker` (RRF / weighted-score / weighted-MRR), mirroring Java `HybridSearchBuilderImpl`.
This tracks the same effort as #563 and its follow-ups. I have the read path (foundation → data layer → PK full-text read → PK hybrid) implemented locally and passing the full test suite; it is gated on the `paimon-ftindex-core` crates.io release before the PRs can be marked ready (a git-only dependency blocks `cargo package`).
### Anything else?
Related: PK-vector read tracking (#514) and draft PR #563. Depends on `apache/paimon-full-text` (`paimon-ftindex-core`) being published to crates.io.
### Willingness to contribute
- [x] I'm willing to submit a PR!
Contributor guide
Research direction
Start with draft PR #563 and the existing primary-key vector read machinery, then trace the shared index/pk source-metadata and bucket-state handling. Review the planned PrimaryKeyFullTextScan, PrimaryKeyFullTextBucketSearch, PrimaryKeyFullTextRead, and PrimaryKeySearchRanker paths. Done means full-text and hybrid primary-key results work behind the fulltext feature and the full test suite passes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- data-engineering, search
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100