infoschema v2 can not handle massy count of partition tables
- Dominant language
- Go
- Stars
- 40.5k
- Forks
- 6.2k
- PR merge metrics
- PR metrics pending
Description
## Enhancement
This is a known issue when we develop infoschema v2
Ref https://github.com/pingcap/tidb/issues/50959
Infoschema v2 aims to handle 1M tables, by keeping only "db name + table name => table id" in memory,
while make "table id => table info" cacheable.
But it does not handle partition table, all partition table info are still in memory.
Recently we found if the user have a lot partition tables, and continuously truncate partition, v2 seems to be regression.
Contributor guide
Research direction
Start by reading the infoschema v2 design in the referenced issue #50959 and tracing how partition table information is retained. Reproduce the regression with many partition tables and repeated partition truncations, then verify that partition metadata no longer remains fully in memory while infoschema v2 still supports the stated scale.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- databases
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100