TiDB Pods stay unhealthy after configuring the `port` parameter in TiDB configuration

Bug Report

What version of Kubernetes are you using?
Client Version: v1.31.1
Kustomize Version: v5.4.2

What version of TiDB Operator are you using?
v1.6.0

What did you do?
We deployed a tidb cluster with 3 replicas of pd, tikv and tidb. After the cluster is fully up and healthy, we changed the spec.tidb.config and set port to 80.
The last replica of TiDB restarted. However, it stayed in the unhealthy state due to the error reported by Readiness probe: Readiness probe failed: dial tcp 10.244.3.8:4000: connect: connection refused. And the operator keeps waiting it and hangs.

The root cause of this bug is similar to https://github.com/pingcap/tidb-operator/issues/6014. The operator hardcodes the port number used to query the readiness of the TiDB here: https://github.com/pingcap/tidb-operator/blob/1867f39610c990467df784268d8a9241667b7083/pkg/manager/member/tidb_member_manager.go#L1229

How to reproduce

1. Deploy a TiDB cluster with TiProxy enabled, for example:

apiVersion: pingcap.com/v1alpha1
kind: TidbCluster
metadata:
  name: test-cluster
spec:
  configUpdateStrategy: RollingUpdate
  enableDynamicConfiguration: true
  helper:
    image: alpine:3.16.0
  pd:
    baseImage: pingcap/pd
    config: "[dashboard]\n  internal-proxy = true\n"
    maxFailoverCount: 0
    mountClusterClientSecret: true
    replicas: 3
    requests:
      storage: 10Gi
  pvReclaimPolicy: Retain
  tidb:
    baseImage: pingcap/tidb
    config: '
      [performance]

      tcp-keep-alive = true
      '
    maxFailoverCount: 0
    replicas: 3
    service:
      externalTrafficPolicy: Local
      type: NodePort
  tikv:
    baseImage: pingcap/tikv
    config: 'log-level = "info"

      '
    maxFailoverCount: 0
    mountClusterClientSecret: true
    replicas: 3
    requests:
      storage: 100Gi
  timezone: UTC
  version: v8.1.0

2. Add port to the spec.tidb.config

apiVersion: pingcap.com/v1alpha1
kind: TidbCluster
metadata:
  name: test-cluster
spec:
  configUpdateStrategy: RollingUpdate
  enableDynamicConfiguration: true
  helper:
    image: alpine:3.16.0
  pd:
    baseImage: pingcap/pd
    config: "[dashboard]\n  internal-proxy = true\n"
    maxFailoverCount: 0
    mountClusterClientSecret: true
    replicas: 3
    requests:
      storage: 10Gi
  pvReclaimPolicy: Retain
  tidb:
    baseImage: pingcap/tidb
    config: 'port = 80

      [performance]

      tcp-keep-alive = true
      '
    maxFailoverCount: 0
    replicas: 3
    service:
      externalTrafficPolicy: Local
      type: NodePort
  tikv:
    baseImage: pingcap/tikv
    config: 'log-level = "info"

      '
    maxFailoverCount: 0
    mountClusterClientSecret: true
    replicas: 3
    requests:
      storage: 100Gi
  timezone: UTC
  version: v8.1.0

What did you expect to see?
We expected to see the tidb rolling update and using the new port.

What did you see instead?
The last replica of tidb restarted. However, it stayed in the unhealthy state due to the error reported by Readiness probe Readiness probe failed: dial tcp 10.244.3.8:4000: connect: connection refused . And the operators kept waiting it and hanged.

Root Cause
After the port of tidb is changed in the configuration, the readiness probe still tries to connect tidb using the default port set in statefulset causing connection failure.

Start in pkg/manager/member/tidb_member_manager.go at the readiness logic around line 1229, then reproduce the configuration change using the TiDB Cluster manifests in the issue. Trace how the configured TiDB port reaches the readiness check; done means a rolling update with port 80 leaves the TiDB replicas healthy instead of probing port 4000.