pingcap / pingcap/tidb

`BATCH ON` response violates MySQL protocol ordering

Open
#60,400 3 comments 0 reactions 1 assignee Claimed by @YangKeao View on GitHub
component/mysql-protocol severity/moderate sig/sql-infra type/bug
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

## Bug Report

Please answer these questions before submitting your issue. Thanks!

### 1. Minimal reproduce step (Required)

1. **Setup:** Have a TiDB instance running.
2. **Schema:** Create a simple table, e.g.:
```sql
CREATE TABLE IF NOT EXISTS batch_test (
id INT PRIMARY KEY,
ts TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
account_id VARBINARY(32),
workchain INT
);
```
3. **Data:** Insert some data that can be targeted by the `BATCH DELETE`:
```sql
-- Insert a row matching the WHERE clause example
INSERT INTO batch_test (id, account_id, workchain) VALUES (1, 0x0020b0a263f2c6fb32d98ad7f8b2a66a269a121109c4349a9943f046677a58ea, 0);
INSERT INTO batch_test (id, account_id, workchain) VALUES (2, 0x0020b0a263f2c6fb32d98ad7f8b2a66a269a121109c4349a9943f046677a58ea, 0);
```
4. **Client:** Use a client capable of sending raw MySQL protocol commands, specifically forcing the `COM_QUERY` protocol path (e.g., `mysql -e "..."` CLI often uses `COM_QUERY`, or a library like Go's `database/sql` without prepared statements, or `sqlx` with a query type forcing non-prepared execution as described in the linked PR).
5. **Query:** Execute a `BATCH` DML statement using the `COM_QUERY` protocol:
```sql
/* Application: tidb_batch_test */
BATCH ON `ts` LIMIT 10
DELETE FROM `batch_test`
WHERE account_id = 0x0020b0a263f2c6fb32d98ad7f8b2a66a269a121109c4349a9943f046677a58ea AND workchain = 0;
```
6. **Observe:** Monitor the raw MySQL network protocol response packets sent by TiDB.

### 2. What did you expect to see? (Required)

Given that the `BATCH` DML command returns structured information about the batch execution ("number of jobs", "job status"), it's expected that when executed via the `COM_QUERY` protocol, TiDB should treat it similarly to a `SELECT` statement that returns results.

Specifically, the expected packet sequence should be:

1. **Packet 1: Column Count:** A packet indicating the number of columns in the result set describing the batch status (e.g., `0x02` for 2 columns).
2. **Packets 2 & 3: Column Definitions:** Standard Column Definition packets describing the columns (e.g., "number of jobs", "job status").
3. **Packet 4: Row(s):** Standard Text Result Set Row packet(s) containing the batch execution details.
4. **Packet 5: EOF / OK Packet:** A final EOF packet (header `0xfe`, potentially interpreted as OK if `CLIENT_DEPRECATE_EOF` is set) marking the end of the result set.

This mirrors the standard MySQL protocol flow observed for a `SELECT 1` statement executed via `COM_QUERY`, which also returns a result set: Column Count -> Column Definition(s) -> Row(s) -> EOF/OK Packet.

Critically, there should **not** be an initial, standalone OK packet sent *before* the result set packets begin.

### 3. What did you see instead (Required)

TiDB sends a sequence of packets that deviates from the standard MySQL protocol for `COM_QUERY` DML:

1. **Packet 1: OK Packet:** An initial OK Packet (header `0x00`) is received, correctly reporting the number of affected rows (e.g., `0x3b` = 59 rows affected in the observed logs). Crucially, the status flags in this packet *do not* include `SERVER_MORE_RESULTS_EXISTS`.
* Example Log Bytes: `003b0002000000` (OK, 59 rows affected, status=2, 0 warnings)

2. **Packet 2: Column Count:** Immediately following the OK packet, *without the client sending anything further*, TiDB sends a single byte representing the column count for a result set (e.g., `0x02` for 2 columns). **This packet reuses the same sequence ID as the preceding OK packet**, which violates the standard protocol sequence ID increment rule.
* Example Log Bytes: `02` (Sequence ID 1, same as OK packet)

3. **Packets 3 & 4: Column Definitions:** Standard Column Definition packets follow, describing the columns for the batch status result set (e.g., "number of jobs", "job status"). Sequence IDs increment correctly from here (e.g., 2, 3).
* Example Log Bytes (Col 1): `036465660000000e6e756d626572206f66206a6f6273000c3f000b000000030000000000`
* Example Log Bytes (Col 2): `036465660000000a6a6f6220737461747573000c2e0000000000fe00001f0000`

4. **Packet 5: Text Row:** A standard Text Result Set Row packet containing the details of the batch execution (e.g., number of jobs processed, status message).
* Example Log Bytes: `033131300d616c6c20737563636565646564` ("110", "all succeeded")

5. **Packet 6: EOF / OK Packet:** A final EOF packet (header `0xfe`, interpreted as OK if `CLIENT_DEPRECATE_EOF` is set) marking the end of the result set. The status flags *do not* contain `SERVER_MORE_RESULTS_EXISTS`.
* Example Log Bytes: `fefc17020002000000` (EOF/OK, 0 warnings with DEPRECATE_EOF, status=2)

**Consequence:** The observed response pattern from TiDB for `BATCH` DML via `COM_QUERY` (an initial OK Packet followed immediately by a full result set detailing batch status) is non-standard and problematic for client libraries:

1. **Hybrid Response Confusion:** Standard MySQL protocol dictates that a `COM_QUERY` response is *either* an OK/ERR packet (for DML) *or* a result set (Column Count -> Defs -> Rows -> EOF/OK, for `SELECT` or similar). TiDB's response mixes these two patterns for a single command, sending both sequentially.
2. **Sequence ID Violation:** The packet immediately following the initial OK packet (the column count `0x02`) reuses the same sequence ID as the OK packet. This violates the protocol rule requiring sequence IDs to increment for each logical packet sent by one party before the other replies, further complicating client-side state management.
3. **Client Library Misinterpretation:** Client libraries (like Rust's `sqlx` when using `pool.execute()`) are typically designed to handle one response pattern or the other per `COM_QUERY`.
* Functions intended for DML (`execute()`) may process the initial OK packet and incorrectly assume the command is finished, leading to errors when unexpected subsequent result set packets arrive.
* Even if the client attempts to process the subsequent packets, the unexpected initial OK packet and the sequence ID reuse can lead to protocol desynchronization, state machine confusion, etc.

### 4. What is your TiDB version? (Required)
```
Release Version: v8.5.0
Edition: Community
Git Commit Hash: d13e52ed6e22cc5789bed7c64c861578cd2ed55b
Git Branch: HEAD
UTC Build Time: 2024-12-18 02:26:06
GoVersion: go1.23.3
Race Enabled: false
Check Table Before Drop: false
Store: tikv
```

---
*Cross-reference to related sqlx fix:* https://github.com/launchbadge/sqlx/pull/3812

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.