dumpling: PD client for GC safepoint ignores TLS credentials on TLS-enabled clusters in v8.5
- Dominant language
- Go
- Stars
- 40.5k
- Forks
- 6.2k
- PR merge metrics
- PR metrics pending
Description
## Bug Report
Please answer these questions before submitting your issue. Thanks!
### 1. Minimal reproduce step (Required)
1. Deploy a classic TiDB v8.5.5 cluster using TiUP with inter-component TLS enabled.
Cluster version: v8.5.5
TLS encryption: enabled
PD: 10.xx.xx.186:12371
TiDB: 10.xx.xx.186:4031
2. Run Dumpling v8.5.5 with the TiUP-generated CA, client certificate, and private key:
tiup dumpling:v8.5.5 \
-h 10.xx.xx..186 \
-P 4031 \
-u root \
-L a.log \
-o /home/tidb/ryl/dumpling_data \
--ca /home/tidb/.tiup/storage/cluster/clusters/tidb-ryl/tls/ca.crt \
--cert /home/tidb/.tiup/storage/cluster/clusters/tidb-ryl/tls/client.crt \
--key /home/tidb/.tiup/storage/cluster/clusters/tidb-ryl/tls/client.pem
3. Dumpling successfully connects to the TiDB SQL port, but its PD client reports:
{
"level": "warn",
"logger": "etcd-client",
"caller": "v3@v3.5.15/retry_interceptor.go:63",
"msg": "retrying of unary invoker failed",
"target": "etcd-endpoints://.../10.2.106.186:12371",
"attempt": 0,
"error": "rpc error: code = DeadlineExceeded desc = latest balancer error: last connection error: connection error: desc = \"error reading server preface: EOF\""
}
4. The same PD endpoint is accessible when the TLS credentials are explicitly passed to pd-ctl:
tiup ctl:v8.5.5 pd \
-u https://10.2.xx.xx:12371 \
--cacert /home/tidb/.tiup/storage/cluster/clusters/tidb-ryl/tls/ca.crt \
--cert /home/tidb/.tiup/storage/cluster/clusters/tidb-ryl/tls/client.crt \
--key /home/tidb/.tiup/storage/cluster/clusters/tidb-ryl/tls/client.pem \
member
### 2. What did you expect to see? (Required)
Dumpling should use the credentials supplied through --ca, --cert, and
--key when creating both:
The TiDB SQL connection.
The PD client used to register and refresh the dumpling_* service GC
safepoint.
No etcd/gRPC connection warning should be emitted, and a dumpling_* entry
### 3. What did you see instead (Required)
The SQL connection uses the provided TLS configuration, but the PD client used
for GC control is created with an empty pd.SecurityOption{}:
pdClient, err := pd.NewClientWithContext(
tctx,
pdAddrs,
pd.SecurityOption{},
)
As a result, Dumpling attempts to connect to a TLS-enabled PD endpoint without
TLS credentials. The connection fails with error reading server preface: EOF,
and Dumpling cannot reliably register or refresh its service GC safepoint.
### 4. What is your TiDB version? (Required)
TiDB cluster version: v8.5.5
Dumpling version: v8.5.5
Dumpling commit: 1fa258b833ff113883beeba40bc130be7ce66610
TiUP cluster version: v1.16.5
Deployment: TiUP, classic cluster
TLS encryption: enabled
OS/Arch: Linux x86_64
Contributor guide
Research direction
Locate the dumpling code that calls pd.NewClientWithContext with pd.SecurityOption{} and trace how --ca, --cert, and --key are passed to the SQL connection. Reproduce against a TLS-enabled PD endpoint using the issue's Dumpling command, then verify that the PD client registers and refreshes the dumpling_* GC safepoint without the etcd/gRPC warning.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, grpc
- Domain
- databases, security
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 68/100