Pod Pending

Question

when deploy tidb cluster, Pod is Pending

env

k3s v1.25.3+k3s1
2 node

command

kubectl create -f https://raw.githubusercontent.com/pingcap/tidb-operator/master/manifests/crd.yaml
helm repo add pingcap https://charts.pingcap.org/
helm repo update
# tidb operator
helm upgrade tidb-operator pingcap/tidb-operator --install -n tidb-operator --create-namespace
# tidb cluster
RELEASE=tidb-cluster TIDB_CLUSTER_NAMESPACE=tidb-cluster-default
helm upgrade ${RELEASE} pingcap/tidb-cluster --install -n ${TIDB_CLUSTER_NAMESPACE} --create-namespace \
    --set monitor.create=false \
    --set pd.replicas=3 \
    --set tikv.replicas=3 \
    --set pd.storageClassName=local-path \
    --set tikv.storageClassName=local-path 

pod info

$ kubectl describe po  tidb-cluster-discovery-577df666bf-jx9rs -n tidb-cluster-default
Name:             tidb-cluster-discovery-577df666bf-jx9rs
Namespace:        tidb-cluster-default
Priority:         0
Service Account:  tidb-cluster-discovery
Node:             <none>
Labels:           app.kubernetes.io/component=discovery
                  app.kubernetes.io/instance=tidb-cluster
                  app.kubernetes.io/managed-by=tidb-operator
                  app.kubernetes.io/name=tidb-cluster
                  pod-template-hash=577df666bf
Annotations:      <none>
Status:           Pending
IP:               
IPs:              <none>
Controlled By:    ReplicaSet/tidb-cluster-discovery-577df666bf
Containers:
  discovery:
    Image:       pingcap/tidb-operator:v1.3.9
    Ports:       10261/TCP, 10262/TCP
    Host Ports:  0/TCP, 0/TCP
    Command:
      /usr/local/bin/tidb-discovery
    Environment:
      MY_POD_NAMESPACE:  tidb-cluster-default (v1:metadata.namespace)
      TZ:                UTC
      TC_NAME:           tidb-cluster
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-w6h8j (ro)
Volumes:
  kube-api-access-w6h8j:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:                      <none>

$ kubectl describe po tidb-cluster-pd-0 -n tidb-cluster-default
Name:             tidb-cluster-pd-0
Namespace:        tidb-cluster-default
Priority:         0
Service Account:  default
Node:             <none>
Labels:           app.kubernetes.io/component=pd
                  app.kubernetes.io/instance=tidb-cluster
                  app.kubernetes.io/managed-by=tidb-operator
                  app.kubernetes.io/name=tidb-cluster
                  controller-revision-hash=tidb-cluster-pd-545864b44b
                  statefulset.kubernetes.io/pod-name=tidb-cluster-pd-0
Annotations:      prometheus.io/path: /metrics
                  prometheus.io/port: 2379
                  prometheus.io/scrape: true
Status:           Pending
IP:               
IPs:              <none>
Controlled By:    StatefulSet/tidb-cluster-pd
Containers:
  pd:
    Image:       pingcap/pd
    Ports:       2380/TCP, 2379/TCP
    Host Ports:  0/TCP, 0/TCP
    Command:
      /bin/sh
      /usr/local/bin/pd_start_script.sh
    Environment:
      NAMESPACE:          tidb-cluster-default (v1:metadata.namespace)
      PEER_SERVICE_NAME:  tidb-cluster-pd-peer
      SERVICE_NAME:       tidb-cluster-pd
      SET_NAME:           tidb-cluster-pd
      TZ:                 UTC
    Mounts:
      /etc/pd from config (ro)
      /etc/podinfo from annotations (ro)
      /usr/local/bin from startup-script (ro)
      /var/lib/pd from pd (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-z6tb2 (ro)
Volumes:
  pd:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  pd-tidb-cluster-pd-0
    ReadOnly:   false
  annotations:
    Type:  DownwardAPI (a volume populated by information about the pod)
    Items:
      metadata.annotations -> annotations
  config:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      tidb-cluster-pd-cfa0d77a
    Optional:  false
  startup-script:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      tidb-cluster-pd-cfa0d77a
    Optional:  false
  kube-api-access-z6tb2:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:                      <none>

tidb operator log

tidb-controller-manager-7ffc5f6fdf-r2zzv.log

Start by reproducing the deployment with the listed k3s, Helm, and kubectl commands, then inspect the discovery and PD pod descriptions, PVCs, and the linked tidb-controller-manager log. Determine why the pods have no assigned node or events, and confirm completion when the TiDB cluster pods schedule and become ready.