planner: cached plan reuses stale analyzer settings for local MATCH ... AGAINST
- Dominant language
- Go
- Stars
- 40.5k
- Forks
- 6.2k
- PR merge metrics
- PR metrics pending
Description
## Bug Report
A cached plan for a locally evaluated `MATCH ... AGAINST` keeps tokenizing by analyzer settings that have since changed, returning rows the current settings say it should not.
Raised in review of #70485.
### Reproduction
```sql
SET @@tidb_enable_local_match_against = ON;
CREATE TABLE t (id INT PRIMARY KEY, body VARCHAR(255));
INSERT INTO t VALUES (1, 'ab cd distributed');
SET GLOBAL innodb_ft_min_token_size = 1;
PREPARE stmt FROM 'SELECT id FROM t WHERE MATCH(body) AGAINST(''+ab'' IN BOOLEAN MODE)';
EXECUTE stmt; -- [1], correct: 'ab' is long enough to be a token
SET GLOBAL innodb_ft_min_token_size = 3;
EXECUTE stmt; -- [1], WRONG: 'ab' is now below the minimum and is dropped,
-- so the query can match nothing
```
The same query run directly, without `PREPARE`, returns the empty set correctly after the change. Only the cached plan is stale.
### Cause
`matchAgainstToLocalBuiltin` in `pkg/planner/core/expression_rewriter.go` resolves the analyzer configuration once via `fulltext.AnalyzerConfigFromSessionVars` and freezes it into `FTSLocalEvalInfo`. Nothing makes the variables it read part of what the plan cache keys on, so a plan outlives the settings that shaped it.
Affects `innodb_ft_min_token_size`, `innodb_ft_max_token_size` and `ngram_token_size`. `innodb_ft_enable_stopword` is also read but is inert today, see #70507.
### Attempted fix that did not work
Calling `sessVars.RecordRelevantOptVar` for each of the four variables alongside the existing `TiDBEnableLocalMatchAgainst` recording does **not** fix it - the reproduction above still returns the stale row. These variables are registered `ScopeGlobal` only, and that mechanism appears not to extend the plan cache key for global-scope variables. Whatever the fix is, it is not that, and a test must actually exercise the prepared path: a direct query re-plans and looks correct either way.
### Notes for a fix
- Test with `innodb_ft_min_token_size`, not `innodb_ft_enable_stopword`: the latter filters nothing today, so a test on it passes whether or not the cache is handled correctly.
- Consider whether the configuration should be resolved at execution time rather than frozen at plan time, which would sidestep the cache-key question entirely - at the cost of the mid-scan stability the current design deliberately buys, so the two need weighing against each other.
Contributor guide
Research direction
Start in pkg/planner/core/expression_rewriter.go at matchAgainstToLocalBuiltin, then trace AnalyzerConfigFromSessionVars and FTSLocalEvalInfo to understand how analyzer settings enter the cached plan. Run the prepared-statement reproduction with innodb_ft_min_token_size, and add a regression test that proves execution after changing the setting does not reuse stale analysis; direct queries should remain correct too.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, sql
- Domain
- backend, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100