pingcap / pingcap/tidb

Wait for drain-clients to complete before shutting down the process after executing the shutdown command

Open
#48,158 0 comments 0 reactions 0 assignees View on GitHub
severity/moderate sig/sql-infra type/bug
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

## Bug Report

Under normal graceful shutdown circumstances (using kill -s TERM), tidb-server waits for up to 15 seconds after closing the TCP port to complete the drain-client operation.

https://github.com/pingcap/tidb/blob/de6ebc057f396079c0380efbdfdabdae45667ec1/cmd/tidb-server/main.go#L933-L951

However, when executing the `shutdown` command, tidb-server currently only waits for 10 seconds before forcibly terminating the process. This behavior is inconsistent with the normal graceful shutdown process and may lead to unexpected disconnects or potential data loss during the client drain phase. It is recommended to align the behavior of the shutdown command with the standard graceful shutdown procedure to ensure consistency.

https://github.com/pingcap/tidb/blob/de6ebc057f396079c0380efbdfdabdae45667ec1/pkg/executor/simple.go#L2765

### 1. Minimal reproduce step (Required)

* session C: `mysql --comments --host 127.0.0.1 --port 4000 -u root -p`

* session A: begin a transaction:
```
(echo "begin;"; for ((i=0; i<100; i++)) do echo "select $i,sleep(1),now();"; done ) | mysql --comments --host 127.0.0.1 --port 4000 -u root -p
```

* session B: `kill -s TERM $tidbpid; date;`

* session C: `mysql> shutdown;`

* session A : (after 10s) `ERROR 2013 (HY000) at line 14: Lost connection to MySQL server during query`

### 2. What did you expect to see? (Required)

Wait for drain-clients to complete before `shutting` down the process after executing the shutdown command.

### 3. What did you see instead (Required)

Wait for drain-clients only 10s before `shutting` down the process after executing the shutdown command.

### 4. What is your TiDB version? (Required)

v7.1.1

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.