TiDB needs double full load schema when restart in million tables scenario
- Dominant language
- Go
- Stars
- 40.5k
- Forks
- 6.2k
- PR merge metrics
- PR metrics pending
Description
## Bug Report
Please answer these questions before submitting your issue. Thanks!
### 1. Minimal reproduce step (Required)
3 miliion tables, restart tidb
Loading schema takes 2m14s, while it needs double load schema. Init stats take 24s. The total restart tidb time is about 5 minutes.
```
[2024/11/27 01:42:32.452 +00:00] [INFO] [domain.go:393] ["full load InfoSchema success"] [isV2=true] [currentSchemaVersion=0] [neededSchemaVersion=5201033] ["elapsed time"=2m14.628440338s]
[2024/11/27 01:42:32.453 +00:00] [INFO] [domain.go:796] ["full load and reset schema validator"]
[2024/11/27 01:42:32.453 +00:00] [WARN] [domain.go:808] ["loading schema takes a long time"] ["take time"=2m14.634251385s]
[2024/11/27 01:42:32.453 +00:00] [WARN] [domain.go:1446] ["loading schema and starting ddl take a long time, we do a new reload"] ["take time"=2m14.634263743s]
...
[2024/11/27 01:45:02.778 +00:00] [INFO] [domain.go:393] ["full load InfoSchema success"] [isV2=true] [currentSchemaVersion=0] [neededSchemaVersion=5201041] ["elapsed time"=2m19.360172359s]
[2024/11/27 01:45:02.780 +00:00] [INFO] [domain.go:796] ["full load and reset schema validator"]
[2024/11/27 01:45:02.780 +00:00] [WARN] [domain.go:808] ["loading schema takes a long time"] ["take time"=2m19.36287633s]
...
[2024/11/27 01:45:08.821 +00:00] [INFO] [bootstrap.go:740] ["complete to load the meta in the lite mode"] [category=stats]
[2024/11/27 01:45:27.096 +00:00] [INFO] [bootstrap.go:746] ["complete to load the histogram in the lite mode"] [category=stats]
[2024/11/27 01:45:27.097 +00:00] [INFO] [domain.go:2482] ["init stats info time"] [lite=true] ["take time"=24.292893795s]
```
### 2. What did you expect to see? (Required)
### 3. What did you see instead (Required)
### 4. What is your TiDB version? (Required)
Release Version: v9.0.0-alpha-59-gc199ddf
Edition: Community
Git Commit Hash: c199ddfcdf9e1f88a243c2b7847d8f3224446004
Git Branch: HEAD
UTC Build Time: 2025-01-07 05:37:06
GoVersion: go1.23.4
Race Enabled: false
Check Table Before Drop: false
Store: tikv
Contributor guide
Research direction
Start in domain.go around the full-load and reload paths reported at lines 393, 796, 808, and 1446, then inspect the related startup flow in bootstrap.go around lines 740-746. Reproduce the restart with 3 million tables and compare the schema and stats loading timings; done means the unnecessary repeated schema load is addressed and restart behavior is verified.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100