pingcap / pingcap/tidb

No needed table scan on stat_meta and stats_histgrams for non-existing tables on i_s.TABLES

Open
#70,480 0 comments 0 reactions 0 assignees View on GitHub
type/enhancement
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

## Enhancement
```sql
SELECT DATA_LENGTH
FROM INFORMATION_SCHEMA.TABLES
WHERE TABLE_SCHEMA = 'definitely_missing'
AND TABLE_NAME = 'definitely_missing';
```
finds no table ids, but `setDataFromTables()` continues with trying to retrieve data from stats_ tables, resulting in a full table scan on stats_meta and stats_histgrams, and only to return a zero result set, since nothing matched.

Also the other extreme, `SELECT * FROM INFORMATION_SCHEMA.TABLES` would extract ALL table ids and partition ids from the whole cluster and then create `WHERE table_id in ()`, which would execute with bad performance.

I propose that if no matching table exists, then just return empty result, and if no WHERE clause (or no schema/tables are listed, or even if the number of table ids is > N%, maybe 20?, no IN clause is used, but do a full table search and filter afterwards in TiDB instead.)

Contributor guide

Open the contributing guide

Research direction

Start by tracing setDataFromTables() for the two INFORMATION_SCHEMA.TABLES queries in the issue and reproduce their access to stats_meta and stats_histgrams. Compare the empty-filter, selective-filter, and unfiltered cases; done means missing tables return an empty result without scanning stats tables, while broad requests avoid an oversized table_id IN clause and preserve correct results.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, sql
Domain
databases, performance
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.