tikv / tikv/pd

Embedded Dashboard: TLS x509 SAN verification fails for loopback-bound TiDB forwarding proxy (fix from tidb-dashboard#1562 not propagated to embedded mode)

Open
#11,078 1 comment 0 reactions 0 assignees View on GitHub
contribution first-time-contributor type/bug
Dominant language
Go
Stars
1.2k
Forks
783
Avg merge
5d 21h
Merged PRs (30d)
36

Description

## Bug Report

PD's embedded Dashboard does not consume the TLS hostname-verification fix from [tidb-dashboard#1562](https://github.com/pingcap/tidb-dashboard/pull/1562) (which resolved [tidb-dashboard#1139](https://github.com/pingcap/tidb-dashboard/issues/1139) for the standalone binary). When cluster TLS is enabled and the server certificate does not include `127.0.0.1` in its IP SANs, login to the embedded Dashboard fails with:

```
x509: cannot validate certificate for 127.0.0.1 because it doesn't contain any IP SANs
```

The standalone `tidb-dashboard` binary works correctly under the same certificate because `buildTLSConfig` in `cmd/tidb-dashboard/main.go` sets `InsecureSkipVerify = true` and supplies a custom `VerifyConnection` callback that verifies the certificate chain only against the trusted CA, skipping hostname matching. The embedded path does not apply this behavior.

### What did you do?

1. Enabled TLS between TiDB components (`require_secure_transport = on`).
2. Configured the embedded Dashboard under `server_configs.pd`:
```yaml
server_configs:
pd:
dashboard.tidb-cacert-path: /tidb-deploy/pd-2379/ssl/ca-cert.pem
dashboard.tidb-cert-path: /tidb-deploy/pd-2379/ssl/client-cert.pem
dashboard.tidb-key-path: /tidb-deploy/pd-2379/ssl/client-key.pem
```
3. Server certificate was issued **without** `subjectAltName` entries (in particular, no `IP:127.0.0.1`).
4. Accessed `https://:2379/dashboard` and attempted to log in.

### What did you expect to see?

Login succeeds, equivalent to the standalone `tidb-dashboard` binary launched with `--tidb-ca / --tidb-cert / --tidb-key`.

### What did you see instead?

Login fails:
```
x509: cannot validate certificate for 127.0.0.1 because it doesn't contain any IP SANs
```

### Root Cause

The Dashboard internally forwards SQL-related HTTP requests to TiDB via a loopback-bound TCP proxy on `127.0.0.1` ([`pkg/tidb/forwarder.go`](https://github.com/pingcap/tidb-dashboard/blob/master/pkg/tidb/forwarder.go)). The TiDB server certificate is presented for the loopback address, so Go's default client-side hostname verification fails when `IP:127.0.0.1` is absent from the SAN extension.

For the standalone binary, PR [tidb-dashboard#1562](https://github.com/pingcap/tidb-dashboard/pull/1562) (merged 2023-07-26) fixed this by overriding the default verification in `buildTLSConfig`:

```go
tlsConfig.InsecureSkipVerify = true
tlsConfig.VerifyConnection = func(state tls.ConnectionState) error {
opts := x509.VerifyOptions{
Intermediates: x509.NewCertPool(),
Roots: tlsConfig.RootCAs,
}
for _, cert := range state.PeerCertificates[1:] {
opts.Intermediates.AddCert(cert)
}
_, err := state.PeerCertificates[0].Verify(opts)
return err
}
```

This change was applied **only** to `cmd/tidb-dashboard/main.go` (standalone entry point). PD's embedded path (`pkg/dashboard` / `DashboardConfig.ToTiDBTLSConfig` in `server/config/config.go`) builds the TLS config with the standard `tlsInfo.ClientConfig()` and never invokes the patched `buildTLSConfig`. As a result, every PD release that embeds Dashboard — including current master and recent LTS branches — regresses on this exact scenario even though the upstream library has the fix.

### Suggested Fix

Pick one (or both):

1. **Promote `buildTLSConfig` to the dashboard library layer** (e.g. `pkg/config` or `pkg/utils/tls`) and have PD's `DashboardConfig.ToTiDBTLSConfig` use it. This is the minimal-change fix and automatically consumes future improvements.
2. **Add a PD config field** so operators can pass a SAN allowlist to the embedded Dashboard:
```yaml
server_configs:
pd:
dashboard.tidb-allowed-names: ["127.0.0.1", "localhost"]
```
mirroring the standalone `--tidb-allowed-names` flag.

### Workaround Until Fixed

Re-issue the TiDB server certificate to include `IP:127.0.0.1` (and `DNS:localhost`) in `subjectAltName`.

### Environment

- PD version: observed on PingKai v7.1.9-0 (kernel ≈ upstream v8.5.x); reproducible on any release whose embedded Dashboard does not carry the equivalent of tidb-dashboard#1562.
- Cluster TLS enabled (`require_secure_transport = on`).
- Server certificate issued without SAN extension.

### Related

- Upstream fix (standalone): https://github.com/pingcap/tidb-dashboard/pull/1562
- Upstream issue: https://github.com/pingcap/tidb-dashboard/issues/1139

Contributor guide

Open the contributing guide

Research direction

Start by comparing cmd/tidb-dashboard/main.go's buildTLSConfig with DashboardConfig.ToTiDBTLSConfig in server/config/config.go and the embedded path under pkg/dashboard. Reproduce the TLS-enabled login failure with a certificate lacking 127.0.0.1 in its IP SANs, then verify that embedded Dashboard login succeeds while the certificate chain remains validated.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
backend, security
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
50/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.