Backup: failed due to external storage internal error.
- Dominant language
- Go
- Stars
- 40.5k
- Forks
- 6.2k
- PR merge metrics
- PR metrics pending
Description
## Bug Report
Please answer these questions before submitting your issue. Thanks!
### 1. Minimal reproduce step (Required)
In https://github.com/pingcap/tidb/issues/34350 we mainly retry the error that TiKV return to br. but in some cases, br alse needs handle the error from external storage.
for example:
br will write meta/lock file to external storage.
And br should tolerate the 5xx error of external storage. and give some fundamental retry.
### 2. What did you expect to see? (Required)
br can retry the 5xx error from external storage.
### 3. What did you see instead (Required)
failed
```
2022/06/17 02:00:48.571 +00:00] [ERROR] [backup.go:41] ["failed to backup"] [error="googleapi: Error 503: We encountered an internal error. Please try again., backendError"] [errorVerbose="googleapi: Error 503: We encountered an internal error. Please try again., backendError\ngithub.com/pingcap/errors.AddStack\n\tgithub.com/pingcap/errors@v0.11.5-0.20201126102027-b0a155152ca3/errors.go:174\ngithub.com/pingcap/errors.Trace\n\tgithub.com/pingcap/errors@v0.11.5-0.20201126102027-b0a155152ca3/juju_adaptor.go:15\ngithub.com/pingcap/br/pkg/task.RunBackup\n\tgithub.com/pingcap/br@/pkg/task/backup.go:245\nmain.runBackupCommand\n\tgithub.com/pingcap/br@/cmd/br/backup.go:40\nmain.newFullBackupCommand.func1\n\tgithub.com/pingcap/br@/cmd/br/backup.go:109\ngithub.com/spf13/cobra.(*Command).execute\n\tgithub.com/spf13/cobra@v1.0.0/command.go:842\ngithub.com/spf13/cobra.(*Command).ExecuteC\n\tgithub.com/spf13/cobra@v1.0.0/command.go:950\ngithub.com/spf13/cobra.(*Command).Execute\n\tgithub.com/spf13/cobra@v1.0.0/command.go:887\nmain.main\n\tgithub.com/pingcap/br@/cmd/br/main.go:56\nruntime.main\n\truntime/proc.go:203\nruntime.goexit\n\truntime/asm_amd64.s:1357"] [stack="main.runBackupCommand\n\tgithub.com/pingcap/br@/cmd/br/backup.go:41\nmain.newFullBackupCommand.func1\n\tgithub.com/pingcap/br@/cmd/br/backup.go:109\ngithub.com/spf13/cobra.(*Command).execute\n\tgithub.com/spf13/cobra@v1.0.0/command.go:842\ngithub.com/spf13/cobra.(*Command).ExecuteC\n\tgithub.com/spf13/cobra@v1.0.0/command.go:950\ngithub.com/spf13/cobra.(*Command).Execute\n\tgithub.com/spf13/cobra@v1.0.0/command.go:887\nmain.main\n\tgithub.com/pingcap/br@/cmd/br/main.go:56\nruntime.main\n\truntime/proc.go:203"]
```
### 4. What is your TiDB version? (Required)
master
Contributor guide
Assessment
This issue has not been assessed yet.