pingcap / pingcap/tidb

TiDB needs double full load schema when restart in million tables scenario

Open
#58,821 1 comment 0 reactions 0 assignees View on GitHub
component/ddl severity/moderate type/bug
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

## Bug Report

Please answer these questions before submitting your issue. Thanks!

### 1. Minimal reproduce step (Required)

3 miliion tables, restart tidb
Loading schema takes 2m14s, while it needs double load schema. Init stats take 24s. The total restart tidb time is about 5 minutes.
```
[2024/11/27 01:42:32.452 +00:00] [INFO] [domain.go:393] ["full load InfoSchema success"] [isV2=true] [currentSchemaVersion=0] [neededSchemaVersion=5201033] ["elapsed time"=2m14.628440338s]
[2024/11/27 01:42:32.453 +00:00] [INFO] [domain.go:796] ["full load and reset schema validator"]
[2024/11/27 01:42:32.453 +00:00] [WARN] [domain.go:808] ["loading schema takes a long time"] ["take time"=2m14.634251385s]
[2024/11/27 01:42:32.453 +00:00] [WARN] [domain.go:1446] ["loading schema and starting ddl take a long time, we do a new reload"] ["take time"=2m14.634263743s]
...
[2024/11/27 01:45:02.778 +00:00] [INFO] [domain.go:393] ["full load InfoSchema success"] [isV2=true] [currentSchemaVersion=0] [neededSchemaVersion=5201041] ["elapsed time"=2m19.360172359s]
[2024/11/27 01:45:02.780 +00:00] [INFO] [domain.go:796] ["full load and reset schema validator"]
[2024/11/27 01:45:02.780 +00:00] [WARN] [domain.go:808] ["loading schema takes a long time"] ["take time"=2m19.36287633s]
...
[2024/11/27 01:45:08.821 +00:00] [INFO] [bootstrap.go:740] ["complete to load the meta in the lite mode"] [category=stats]
[2024/11/27 01:45:27.096 +00:00] [INFO] [bootstrap.go:746] ["complete to load the histogram in the lite mode"] [category=stats]
[2024/11/27 01:45:27.097 +00:00] [INFO] [domain.go:2482] ["init stats info time"] [lite=true] ["take time"=24.292893795s]
```
### 2. What did you expect to see? (Required)

### 3. What did you see instead (Required)

### 4. What is your TiDB version? (Required)

Release Version: v9.0.0-alpha-59-gc199ddf
Edition: Community
Git Commit Hash: c199ddfcdf9e1f88a243c2b7847d8f3224446004
Git Branch: HEAD
UTC Build Time: 2025-01-07 05:37:06
GoVersion: go1.23.4
Race Enabled: false
Check Table Before Drop: false
Store: tikv

Contributor guide

Open the contributing guide

Research direction

Start in domain.go around the full-load and reload paths reported at lines 393, 796, 808, and 1446, then inspect the related startup flow in bootstrap.go around lines 740-746. Reproduce the restart with 3 million tables and compare the schema and stats loading timings; done means the unnecessary repeated schema load is addressed and restart behavior is verified.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.