pingcap / pingcap/tidb

br: flaky int test br_cache_table

Open
#58,639 0 comments 0 reactions 0 assignees View on GitHub
component/br component/test may-affects-5.4 may-affects-6.1 may-affects-6.5 may-affects-7.1 may-affects-7.5 may-affects-8.1 may-affects-8.5 severity/major type/bug
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

## Bug Report

found checksum mismatch at the end of the test

```
[2024/12/31 12:10:16.758 +08:00] [ERROR] [restore.go:81] ["failed to restore"] [error="failed to validate checksum: [BR:Restore:ErrRestoreChecksumMismatch]restore checksum mismatch"] [errorVerbose="[BR:Restore:ErrRestoreChecksumMismatch]restore checksum mismatch\nfailed to validate checksum\ngithub.com/pingcap/tidb/br/pkg/restore/snap_client.(*SnapClient).execAndValidateChecksum\n\t/home/jenkins/agent/workspace/pingcap/tidb/pull_br_integration_test/tidb/br/pkg/restore/snap_client/client.go:1129\ngithub.com/pingcap/tidb/br/pkg/restore/snap_client.(*SnapClient).GoValidateChecksum.func1\n\t/home/jenkins/agent/workspace/pingcap/tidb/pull_br_integration_test/tidb/br/pkg/restore/snap_client/pipeline_items.go:122\ngithub.com/pingcap/tidb/br/pkg/restore/snap_client.concurrentHandleTablesCh.func2\n\t/home/jenkins/agent/workspace/pingcap/tidb/pull_br_integration_test/tidb/br/pkg/restore/snap_client/pipeline_items.go:92\ngolang.org/x/sync/errgroup.(*Group).Go.func1\n\t/go/pkg/mod/golang.org/x/sync@v0.10.0/errgroup/errgroup.go:78\nruntime.goexit\n\t/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.23.4.linux-amd64/src/runtime/asm_amd64.s:1700"] [stack="github.com/pingcap/tidb/br/cmd/br.runRestoreCommand\n\t/home/jenkins/agent/workspace/pingcap/tidb/pull_br_integration_test/tidb/br/cmd/br/restore.go:81\ngithub.com/pingcap/tidb/br/cmd/br.newDBRestoreCommand.func1\n\t/home/jenkins/agent/workspace/pingcap/tidb/pull_br_integration_test/tidb/br/cmd/br/restore.go:200\ngithub.com/spf13/cobra.(*Command).execute\n\t/go/pkg/mod/github.com/spf13/cobra@v1.8.1/command.go:985\ngithub.com/spf13/cobra.(*Command).ExecuteC\n\t/go/pkg/mod/github.com/spf13/cobra@v1.8.1/command.go:1117\ngithub.com/spf13/cobra.(*Command).Execute\n\t/go/pkg/mod/github.com/spf13/cobra@v1.8.1/command.go:1041\ngithub.com/pingcap/tidb/br/cmd/br.main\n\t/home/jenkins/agent/workspace/pingcap/tidb/pull_br_integration_test/tidb/br/cmd/br/main.go:36\ngithub.com/pingcap/tidb/br/cmd/br.TestRunMain.func1\n\t/home/jenkins/agent/workspace/pingcap/tidb/pull_br_integration_test/tidb/br/cmd/br/main_test.go:73"]
Error: failed to validate checksum: [BR:Restore:ErrRestoreChecksumMismatch]restore checksum mismatch
[2024/12/31 12:10:16.758 +08:00] [ERROR] [main.go:38] ["br failed"] [error="failed to validate checksum: [BR:Restore:ErrRestoreChecksumMismatch]restore checksum mismatch"] [errorVerbose="[BR:Restore:ErrRestoreChecksumMismatch]restore checksum mismatch\nfailed to validate checksum\ngithub.com/pingcap/tidb/br/pkg/restore/snap_client.(*SnapClient).execAndValidateChecksum\n\t/home/jenkins/agent/workspace/pingcap/tidb/pull_br_integration_test/tidb/br/pkg/restore/snap_client/client.go:1129\ngithub.com/pingcap/tidb/br/pkg/restore/snap_client.(*SnapClient).GoValidateChecksum.func1\n\t/home/jenkins/agent/workspace/pingcap/tidb/pull_br_integration_test/tidb/br/pkg/restore/snap_client/pipeline_items.go:122\ngithub.com/pingcap/tidb/br/pkg/restore/snap_client.concurrentHandleTablesCh.func2\n\t/home/jenkins/agent/workspace/pingcap/tidb/pull_br_integration_test/tidb/br/pkg/restore/snap_client/pipeline_items.go:92\ngolang.org/x/sync/errgroup.(*Group).Go.func1\n\t/go/pkg/mod/golang.org/x/sync@v0.10.0/errgroup/errgroup.go:78\nruntime.goexit\n\t/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.23.4.linux-amd64/src/runtime/asm_amd64.s:1700"] [stack="github.com/pingcap/tidb/br/cmd/br.main\n\t/home/jenkins/agent/workspace/pingcap/tidb/pull_br_integration_test/tidb/br/cmd/br/main.go:38\ngithub.com/pingcap/tidb/br/cmd/br.TestRunMain.func1\n\t/home/jenkins/agent/workspace/pingcap/tidb/pull_br_integration_test/tidb/br/cmd/br/main_test.go:73"]

```

mentioned
```
[2024/12/31 12:10:00.530 +08:00] [WARN] [client.go:263] ["Keyspace BR is not supported in this cluster, fallback to legacy restore"] [error="rpc error: code = Unavailable desc = not leader"] [errorVerbose="rpc error: code = Unavailable desc = not leader\ngithub.com/tikv/pd/client.(*client).respForErr\n\t/go/pkg/mod/github.com/tikv/pd/client@v0.0.0-20241220053006-461b86adc78d/client.go:1358\ngithub.com/tikv/pd/client.(*client).GetAllStores\n\t/go/pkg/mod/github.com/tikv/pd/client@v0.0.0-20241220053006-461b86adc78d/client.go:986\ngithub.com/pingcap/tidb/br/pkg/version.CheckClusterVersion\n\t/home/jenkins/agent/workspace/pingcap/tidb/pull_br_integration_test/tidb/br/pkg/version/version.go:90\ngithub.com/pingcap/tidb/br/pkg/restore/snap_client.
```

probably has something to do with it? Need to repro locally

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.