matrixorigin / matrixorigin/matrixone
[Performance]: safely push candidate LIMIT for exact FULLTEXT2 pure-MUST filters
- Dominant language
- Go
- Stars
- 1.9k
- Forks
- 311
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 768
Description
## Background
FULLTEXT2 exact integer primary-key membership can safely bound the internal candidate stream for a literal pure-MUST Boolean query. Without the bound, a query with `LIMIT/OFFSET` may materialize many more candidates than the final page needs. Approximate Bloom membership and richer Boolean shapes cannot use this optimization because false positives or later Boolean evaluation can under-fill the page.
## Proposed scope
- Push `LIMIT + OFFSET` into the FULLTEXT2 table function only for one literal pure-MUST Boolean MATCH stream with an exact integer prefilter and pushdown enabled.
- Preserve the existing safe limit behavior when there is no residual filter.
- Keep VARCHAR/UUID/Bloom membership, prepared patterns, multiple MATCH streams, phrase, prefix, SHOULD, MUST-NOT, and other complex shapes unbounded at the table function.
## Correctness evidence
- BIGINT exact membership with `LIMIT 2 OFFSET 1` receives candidate limit 3 and returns the same page as pushdown OFF.
- A high-scoring filtered-out document does not cause under-fill.
- VARCHAR/Bloom has no table-function candidate limit and matches pushdown OFF.
- Complex Boolean, phrase, prefix, and MUST-NOT plans have no candidate limit.
- The dedicated FULLTEXT2 BVT passed 24/24 on an isolated candidate standalone.
## Non-goals
- No WAND, F1B, term statistics, filtered-scan reuse, INCLUDE fields, SQL syntax, index format, CDC, or runtime-filter wire-protocol changes.
- No claim of a search-kernel speedup; the observed benefit is reducing downstream candidates only.
## QA decision
- QA required: yes
- Reason: this is a planner/table-function correctness boundary involving LIMIT semantics and exact versus approximate membership.
- Production entrypoint: SQL `MATCH ... AGAINST (... IN BOOLEAN MODE)` with an exact integer WHERE prefilter and literal LIMIT/OFFSET.
- Automated terminal coverage: typed-plan tests, under-fill fixture, FULLTEXT2 table-function tests, and the dedicated distributed BVT.
- Remaining validation: exact-head CI and reviewer verification of the plan-shape boundary.
Contributor guide
Assessment
This issue has not been assessed yet.