pingcap / pingcap/tidb

br: volume snapshot backup looks stuck (10 minutes timeout) when tikv version is lower (<v6.3.0)

Open
#41,966 0 comments 0 reactions 1 assignee Claimed by @fengou1 View on GitHub
component/br severity/moderate type/bug
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

## Bug Report

Please answer these questions before submitting your issue. Thanks!

### 1. Minimal reproduce step (Required)
follow https://docs.pingcap.com/zh/tidb-in-kubernetes/stable/configure-a-tidb-cluster to configure the tidb cluster with
1. cluster version = v6.5.0
2. tidb version = v6.1.0
3. tikv version = v6.1.0
```yaml
apiVersion: pingcap.com/v1alpha1
kind: TidbCluster
metadata:
name: basic
namespace: tidb-cluster
spec:
version: v6.5.0
timezone: UTC
configUpdateStrategy: RollingUpdate
pvReclaimPolicy: Delete
schedulerName: default-scheduler
topologySpreadConstraints:
- topologyKey: topology.kubernetes.io/zone
enableDynamicConfiguration: true
helper:
image: alpine:3.16.0
pd:
baseImage: pingcap/pd
maxFailoverCount: 0
replicas: 1
requests:
storage: "30Gi"
config: |
[dashboard]
internal-proxy = true
[replication]
location-labels = ["topology.kubernetes.io/zone", "kubernetes.io/hostname"]
max-replicas = 3
nodeSelector:
dedicated: pd
tolerations:
- effect: NoSchedule
key: dedicated
operator: Equal
value: pd
affinity:
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: app.kubernetes.io/component
operator: In
values:
- pd
topologyKey: kubernetes.io/hostname
tikv:
baseImage: pingcap/tikv
maxFailoverCount: 0
replicas: 3
requests:
storage: "350Gi"
storageClassName: gp3
config: {}
nodeSelector:
dedicated: tikv
tolerations:
- effect: NoSchedule
key: dedicated
operator: Equal
value: tikv
affinity:
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: app.kubernetes.io/component
operator: In
values:
- tikv
topologyKey: kubernetes.io/hostname
version: "v6.1.0"
tidb:
baseImage: pingcap/tidb
maxFailoverCount: 0
replicas: 1
service:
annotations:
service.beta.kubernetes.io/aws-load-balancer-cross-zone-load-balancing-enabled: 'true'
service.beta.kubernetes.io/aws-load-balancer-type: nlb
service.beta.kubernetes.io/aws-load-balancer-scheme: internal
service.beta.kubernetes.io/aws-load-balancer-internal: "true"
exposeStatus: true
externalTrafficPolicy: Local
type: LoadBalancer
config: |
split-table = false
[performance]
tcp-keep-alive = true
annotations:
tidb.pingcap.com/sysctl-init: "true"
podSecurityContext:
sysctls:
- name: net.ipv4.tcp_keepalive_time
value: "300"
- name: net.ipv4.tcp_keepalive_intvl
value: "75"
- name: net.core.somaxconn
value: "32768"
separateSlowLog: true
nodeSelector:
dedicated: tidb
tolerations:
- effect: NoSchedule
key: dedicated
operator: Equal
value: tidb
affinity:
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: app.kubernetes.io/component
operator: In
values:
- tidb
topologyKey: kubernetes.io/hostname

```
follow https://docs.pingcap.com/tidb-in-kubernetes/stable/backup-to-aws-s3-by-snapshot to do the backup

### 2. What did you expect to see? (Required)
Backup shall failure immediately

### 3. What did you see instead (Required)
backup is retry with error Unimplement of check admin phase.
![BLQGuOXLsx](https://user-images.githubusercontent.com/85682690/223295572-de4de3a0-2b2f-4cab-82da-8b35e8fe497f.jpg)

### 4. What is your TiDB version? (Required)
v6.1.0

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.