pingcap / pingcap/tidb

planner: cached plan reuses stale analyzer settings for local MATCH ... AGAINST

Open
#70,512 0 comments 0 reactions 0 assignees View on GitHub
severity/moderate sig/planner type/bug
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

## Bug Report

A cached plan for a locally evaluated `MATCH ... AGAINST` keeps tokenizing by analyzer settings that have since changed, returning rows the current settings say it should not.

Raised in review of #70485.

### Reproduction

```sql
SET @@tidb_enable_local_match_against = ON;
CREATE TABLE t (id INT PRIMARY KEY, body VARCHAR(255));
INSERT INTO t VALUES (1, 'ab cd distributed');

SET GLOBAL innodb_ft_min_token_size = 1;
PREPARE stmt FROM 'SELECT id FROM t WHERE MATCH(body) AGAINST(''+ab'' IN BOOLEAN MODE)';
EXECUTE stmt; -- [1], correct: 'ab' is long enough to be a token

SET GLOBAL innodb_ft_min_token_size = 3;
EXECUTE stmt; -- [1], WRONG: 'ab' is now below the minimum and is dropped,
-- so the query can match nothing
```

The same query run directly, without `PREPARE`, returns the empty set correctly after the change. Only the cached plan is stale.

### Cause

`matchAgainstToLocalBuiltin` in `pkg/planner/core/expression_rewriter.go` resolves the analyzer configuration once via `fulltext.AnalyzerConfigFromSessionVars` and freezes it into `FTSLocalEvalInfo`. Nothing makes the variables it read part of what the plan cache keys on, so a plan outlives the settings that shaped it.

Affects `innodb_ft_min_token_size`, `innodb_ft_max_token_size` and `ngram_token_size`. `innodb_ft_enable_stopword` is also read but is inert today, see #70507.

### Attempted fix that did not work

Calling `sessVars.RecordRelevantOptVar` for each of the four variables alongside the existing `TiDBEnableLocalMatchAgainst` recording does **not** fix it - the reproduction above still returns the stale row. These variables are registered `ScopeGlobal` only, and that mechanism appears not to extend the plan cache key for global-scope variables. Whatever the fix is, it is not that, and a test must actually exercise the prepared path: a direct query re-plans and looks correct either way.

### Notes for a fix

- Test with `innodb_ft_min_token_size`, not `innodb_ft_enable_stopword`: the latter filters nothing today, so a test on it passes whether or not the cache is handled correctly.
- Consider whether the configuration should be resolved at execution time rather than frozen at plan time, which would sidestep the cache-key question entirely - at the cost of the mid-scan stability the current design deliberately buys, so the two need weighing against each other.

Contributor guide

Open the contributing guide

Research direction

Start in pkg/planner/core/expression_rewriter.go at matchAgainstToLocalBuiltin, then trace AnalyzerConfigFromSessionVars and FTSLocalEvalInfo to understand how analyzer settings enter the cached plan. Run the prepared-statement reproduction with innodb_ft_min_token_size, and add a regression test that proves execution after changing the setting does not reuse stale analysis; direct queries should remain correct too.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, sql
Domain
backend, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.