pingcap / pingcap/tidb

[br] BR full restore can report success after a schema metadata read failure

Open
#70,128 2 comments 0 reactions 0 assignees View on GitHub
affects-7.5 affects-8.1 affects-8.5 component/br found-by-ai severity/critical type/bug
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

## Bug Report

### 1. Minimal reproduce step (Required)

**Environment**

TiDB 05b396fb66; default backupmeta V2; one TiDB/PD/real TiKV; MDL ON; no failpoints

**Steps**

On a dedicated fresh TiDB/TiKV cluster, create a named canary table with two rows and run BR full backup with the default V2 metadata layout. Copy the backup, move its referenced backupmeta.schema.* object out of the copy, drop the canary schema, and repeatedly run BR restore full from the corrupted copy without explicit filters. Check the process exit code, Full Restore summary, loaded database count, and exact destination canary rows. A focused unit test can repeat LoadBackupTables 500 times per run with a V2 SchemaIndex that references a missing object.

### 2. What did you expect to see? (Required)

Every terminal schema metadata read error makes BR restore return nonzero with the storage error. A successful full restore must contain the named canary schema, table, and two rows.

### 3. What did you see instead? (Required)

Current master returned exit code 0 on the first product attempt, logged databases=0, nothing to restore, snapshot restore success, and Full Restore success summary with Size=0. The canary schema and table remained absent. In a 5000-call unit amplification, 55 calls returned nil error with an empty database set.

### 4. What is your TiDB version? (Required)

TiDB 05b396fb66; default backupmeta V2; one TiDB/PD/real TiKV; MDL ON; no failpoints

Likely root cause and fix direction

**Likely root cause**

ReadSchemasFiles sends read errors through a buffered errCh and closes the parsed-result channel separately. receiveBatch selects between errCh and ch; when both the terminal error and closed ch are ready, selecting closed ch returns nil. LoadBackupTables publishes the empty set, and unfiltered full restore has no manifest-count check before its explicit empty-restore success path.

**Fix direction**

Join terminal result and error ownership. At minimum, when result close is observed, surface any error published before close. Prefer a single terminal result or an errgroup/owner structure that makes successful completion impossible until all producers are joined. Audit the sibling file metadata loop and cancellation path as part of the fix.

Contributor guide

Open the contributing guide

Research direction

Start with ReadSchemasFiles, receiveBatch, and LoadBackupTables, then run the focused unit test that repeats the missing-schema-object case. Trace how terminal read errors and closed result channels are handled, including the sibling file metadata loop and cancellation path. Done means every terminal metadata error reaches restore, while a successful full restore contains the expected schema, table, and rows.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.