tikv / tikv/pd

Need readiness check endpoint for PD

Open
#5,658 6 comments 0 reactions 0 assignees View on GitHub
type/feature-request
Dominant language
Go
Stars
1.2k
Forks
783
Avg merge
5d 21h
Merged PRs (30d)
36

Description

## Feature Request

### Describe your feature request related problem

When doing a rolling update for a PD cluster, we need to turn down a PD instance, then bring up a PD instance with the updated config, and repeat this process for all PD instances one at a time. Typically we use tidb-operator for automating this, and a PD instance is considered "Running" as long as the pod is up for some duration. However, it could be the PD instance is still picking up with the leader and receiving etcd raft logs, and cannot serve the request at the moment. If let's say we have a deployment of PDs that has 3 replicas, the first two pods are updated but not ready for serving new requests, then at this point the PD is unavailable since the remaining PD cannot write quorum (as the other two are still syncing etcd).

### Describe the feature you'd like

It would be helpful if PD instance can expose info (maybe via a restful endpoint) whether its internal data is synced with leader and ready for serving new requests.

### Describe alternatives you've considered

An alternative approach in tidb-operator scenario is to add a init container in PD's pod spec that sleeps for certain time. Before sleep is done, the Pod will stuck in `Init` state so the rolling update process will wait til the sleep completes. Hopefully, during this time, the etcd data for that PD instance is synced.

### Teachability, Documentation, Adoption, Migration Strategy

The endpoint should be something new, and since old system is depending on it, we don't need to worry about the migration.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.