pingcap / pingcap/tidb

infoschema v2 can not handle massy count of partition tables

Open
#67,510 0 comments 0 reactions 0 assignees View on GitHub
sig/sql-infra type/enhancement
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

## Enhancement

This is a known issue when we develop infoschema v2

Ref https://github.com/pingcap/tidb/issues/50959

Infoschema v2 aims to handle 1M tables, by keeping only "db name + table name => table id" in memory,
while make "table id => table info" cacheable.

But it does not handle partition table, all partition table info are still in memory.

Recently we found if the user have a lot partition tables, and continuously truncate partition, v2 seems to be regression.

Image

Contributor guide

Open the contributing guide

Research direction

Start by reading the infoschema v2 design in the referenced issue #50959 and tracing how partition table information is retained. Reproduce the regression with many partition tables and repeated partition truncations, then verify that partition metadata no longer remains fully in memory while infoschema v2 still supports the stated scale.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
databases
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.