pingcap / pingcap/tidb

PD/DDL: Missing necessary metrics from etcd to identify the root cause of High 'Update Self Version Duration' when modifying column

Open
#61,623 2 comments 0 reactions 0 assignees View on GitHub
component/pd severity/moderate type/bug
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

## Bug Report

Please answer these questions before submitting your issue. Thanks!

### 1. Minimal reproduce step (Required)

Run continuous column modification DDLs on a TiDB cluster.

### 2. What did you expect to see? (Required)

The "Update Self Version Duration" metric in the DDL subsystem should remain low and stable.

### 3. What did you see instead (Required)

The "Update Self Version Duration" metric consistently peaks at around 9 minutes.

![Image](https://github.com/user-attachments/assets/cbb2018d-042a-4f57-bff3-98ae17bbb495)

There is no clear cause for this delay based on etcd metrics.

![Image](https://github.com/user-attachments/assets/6dfb187f-92f8-43d4-b286-7ff8cbbba083)

![Image](https://github.com/user-attachments/assets/8306609f-be63-4f58-a60f-a547c2902edb)

Note: etcd’s disk WAL fsync rate does increase during this period, but it shouldn't be enough to cause such a dramatic delay. And the continues MVCC and other gRPC durations remain normal and stable.

### 4. What is your TiDB version? (Required)

v7.5.6

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.