executor: push down table_id/index_id filters for TIFLASH_INDEXES system table
- Dominant language
- Go
- Stars
- 40.5k
- Forks
- 6.2k
- PR merge metrics
- PR metrics pending
Description
## Enhancement
`ADD VECTOR INDEX` progress polling against `information_schema.tiflash_indexes` should push `table_id` / `index_id` predicates down to TiFlash.
## Problem
When a cluster has many TiFlash vector-index tables, the DDL progress checker queries `information_schema.tiflash_indexes` for a specific table/index. Today `TiFlashSystemTableExtractor` only extracts predicates such as TiFlash instance, database, and table name for TiFlash system-table requests. It does not extract `table_id` / `index_id`, so the TiDB retriever sends a broad TiFlash system-table query.
At 100k vector-index table scale, this makes progress polling scale with all TiFlash local-index metadata instead of the target table/index.
Recent profiling evidence:
- Capture `2026-05-31 17:55 CST`: about 87,762 vector-index tables were ready and `ADD VECTOR INDEX` was still running.
- Capture `2026-06-01 08:33 CST`: about 99.9k vector-index tables were ready.
- TiDB CPU profile: `TiFlashSystemTableRetriever.dataForTiFlashSystemTables` was a major TiDB CPU consumer; `encoding/json.Unmarshal` and GC were also visible.
- TiDB allocation profile: `TiFlashSystemTableRetriever.dataForTiFlashSystemTables` and `LoadTiFlashReplicaInfo` allocated heavily during progress polling.
- TiFlash CPU profiles showed the companion bottleneck in `StorageSystemDTLocalIndexes::read` / `DeltaMergeStore::getLocalIndexStats`.
This issue tracks the TiDB side of a two-layer fix. TiFlash should also prune early before calling `getLocalIndexStats()`.
## Proposed change
- Extend `TiFlashSystemTableExtractor` for `information_schema.tiflash_indexes` / `TableTiFlashIndexes` to extract `table_id` and `index_id` predicates.
- Support at least equality and `IN` predicates, including conjunctions with the existing extracted predicates.
- Carry the extracted `table_id` / `index_id` filters into the TiFlash system-table request SQL so TiFlash can prune by exact table/index.
- Keep behavior unchanged for other TiFlash system tables.
- Handle contradiction / empty ranges by skipping the TiFlash request when possible.
- Add tests for equality, `IN`, conjunctions, and non-pushable predicates.
## Acceptance criteria
- `EXPLAIN` or equivalent executor diagnostics show that `table_id` / `index_id` filters are extracted for `information_schema.tiflash_indexes`.
- `ADD VECTOR INDEX` progress polling sends a targeted TiFlash system-table query containing the relevant `table_id` / `index_id` filter.
- Combined with the TiFlash-side pruning change, progress polling cost becomes proportional to the matched table/index instead of all TiFlash vector-index tables.
Contributor guide
Research direction
Locate TiFlashSystemTableExtractor and the TableTiFlashIndexes handling, then trace how extracted predicates reach TiFlashSystemTableRetriever.dataForTiFlashSystemTables. Check existing extractor and executor diagnostics tests first; done means equality, IN, conjunction, non-pushable, and contradiction cases are covered and the generated request retains table_id/index_id filters.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- backend, databases
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100