I create a cluster with 2M users, and use BR to backup / restore it, meet error
- Dominant language
- Go
- Stars
- 40.5k
- Forks
- 6.2k
- PR merge metrics
- PR metrics pending
Description
## Bug Report
Please answer these questions before submitting your issue. Thanks!
### 1. Minimal reproduce step (Required)
I create a cluster with 2M users:
```
mysql> use mysql;
Database changed
mysql> select count(*) from user;
+----------+
| count(*) |
+----------+
| 2494895 |
+----------+
1 row in set (0.71 sec)
```
Use br to backup:
```
tiup br:nightly backup full --storage "local:///tmp/backup"
```
and restore it:
```
tiup playground nightly --host 0.0.0.0 --tiflash 0 --db.binpath ./bin/tidb-server
tiup br:nightly restore full --storage "local:///tmp/backup"
```
### 2. What did you expect to see? (Required)
Success
### 3. What did you see instead (Required)
```
tiup br:nightly restore full --storage "local:///tmp/backup"
Starting component br: /home/genius/.tiup/components/br/v9.0.0-alpha-nightly/br restore full --storage local:///tmp/backup
Detail BR log in /tmp/br.log.2025-02-20T15.21.56+0800
Split&Scatter Regions <---------------------------------------------------------------------------------------> 100.00%
Download&Ingest SST <-----------------------------------------------------------------------------------------> 100.00%
Restore Pipeline <--------------------------------------------------------------------------------------------> 100.00%
[2025/02/20 15:22:19.354 +08:00] [INFO] [collector.go:77] ["Full Restore failed summary"] [total-ranges=240] [ranges-succeed=240] [ranges-failed=0] [merge-ranges=798.965µs] [split-regions=125.662394ms] [restore-files=2.746380622s] [restore-pipeline=5.142510093s] [default-CF-files=47] [write-CF-files=193] [split-keys=9]
Error: error during merging temporary tables into system tables, table: global_priv: [BR:Common:ErrUnknown]failed to execute REPLACE INTO mysql.global_priv(host,user,priv) SELECT host,user,priv FROM __tidb_br_temporary_mysql.global_priv;: [kv:8004]Transaction is too large, size: 104857618
Error: run /home/genius/.tiup/components/br/v9.0.0-alpha-nightly/br (wd:/home/genius/.tiup/data/UdKIDAX) failed: exit status 1
```
It use a single `REPLACE INTO ` for the mysql tables, when there are 2M users, the transaction is too large.
### 4. What is your TiDB version? (Required)
master 2025-02-20
Contributor guide
Research direction
Start by reproducing the documented tiup br:nightly backup and restore commands with a large mysql.user table, then trace the restore step that merges into mysql.global_priv and the temporary table. Done means the full restore completes successfully without the 104857618-byte transaction-too-large error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, mysql
- Domain
- databases, distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100