MQ consumer can drop valid DDLSpan-only DDLs before sorting

What did you do?

Reviewed the MQ consumer DDL buffering path in cmd/kafka-consumer/writer.go and cmd/pulsar-consumer/writer.go.

appendDDL() rejects a DDL before enqueue when its commitTs is smaller than the maximum commitTs previously seen for any key returned by getBlockTableIDs().

For CREATE SCHEMA and independent CREATE TABLE, the blocked-table set can collapse to the shared DDLSpanTableID. When a newer DDLSpan-only DDL for object A is received before an older but still valid DDLSpan-only DDL for object B, the older DDL can be dropped here before Write() gets a chance to sort ddlList by commitTs.

Once dropped in appendDDL(), the later commitTs sort cannot recover it.

What did you expect to see?

Cross-object DDLSpan-only DDLs should both remain queued and then be ordered by commitTs.

Stale-drop protection should only reject truly stale DDLs for the same logical object.

What did you see instead?

The consumer reuses the blocked-table set as the stale-drop identity, so the shared DDLSpanTableID acts as a global key.

That lets unrelated CREATE SCHEMA / independent CREATE TABLE DDLs suppress each other before enqueue.

This becomes more visible once these DDLs are allowed to bypass watermark gating, because commitTs ordering then depends on ddlList sorting and dropped events never reach that stage.

Versions of the cluster

Upstream TiDB cluster version (execute SELECT tidb_version(); in a MySQL client):

not captured; issue identified by code inspection of the MQ consumer path

Upstream TiKV version (execute tikv-server --version):

not captured; issue identified by code inspection of the MQ consumer path

TiCDC version (execute cdc version):

not captured; issue identified on the current MQ consumer branch around the PR #4035 follow-up path