matrixorigin / matrixorigin/matrixone
[Performance]: keep FULLTEXT2 cache read-only across CDC updates
- Dominant language
- Go
- Stars
- 1.9k
- Forks
- 311
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 768
Description
## Background
PR #27461 added FULLTEXT2 cold-load observability and cache invalidation lifecycle changes. PR #27462 was stacked on it and added reusable immutable bases plus incremental tail/delete loading across cache generations.
The required cache contract is simpler and stricter: a cached FULLTEXT2 index is read-only for its entire lifetime. Visibility delay after CDC is accepted. CDC must not evict, refresh, or incrementally update the cached search index.
## Problem
When CDC evicts the warm entry, the next search creates and loads a replacement cache entry. The load is serialized by the cache entry write lock, so concurrent searches for that index can wait for the refresh. Triggering this after every incremental update can therefore block concurrent searches.
Reverting only #27462 is insufficient. #27461 is its prerequisite and also changes the invalidation lifecycle, while the older CDC consumer already calls `Cache.Remove` after a successful flush.
## Required behavior
- A cached FULLTEXT2 search index is immutable and read-only until ordinary stale/TTL eviction.
- CDC persists tail frames but does not evict, mutate, or incrementally refresh the cached search index.
- Visibility delay until the accepted stale/TTL boundary is allowed.
- Concurrent warm searches do not wait for a CDC-triggered cache refresh.
- Keep SQL, DDL, index format, ranking, and CDC wire protocol unchanged.
- Preserve deterministic mmap/temp-file cleanup at ordinary eviction, shutdown, MERGE, REBUILD, errors, and cancellation.
## Immediate action
- Revert #27462 and #27461.
- Remove the pre-existing CDC-triggered `Cache.Remove` path so reverting the two PRs does not leave immediate invalidation enabled.
- Keep cache replacement on the existing stale/TTL lifecycle.
- Preserve the generic vector-cache lifecycle fixes already shared by other index algorithms; only the FULLTEXT2 observability/reuse behavior is reverted.
The current-main revert must also retain safe failed-load cleanup and partial tail-load cleanup, because those are independent correctness requirements rather than incremental-cache reuse behavior.
## Validation
- FULLTEXT2 and vector-index cache package tests, including race coverage.
- CDC plus concurrent-search regression proving warm cached readers are not refreshed or blocked by CDC.
- TTL/stale eviction, MERGE, REBUILD, restart, cancellation, and cleanup coverage.
- Production-path FULLTEXT2 BVT where the cache contract is SQL-reachable; otherwise document the precise BVT boundary and equivalent deterministic regression.
## Related
- #27445
- #27461
- #27462
## QA decision
QA required: yes. This changes CDC visibility timing, cache invalidation, concurrent search behavior, and mmap/temp-file lifecycle. Deployment validation must cover CDC, stale/TTL expiry, MERGE, REBUILD, restart, and concurrent searches.
Contributor guide
Assessment
This issue has not been assessed yet.