TiCDC v8.5.7 may panic when batch-dml-enable=true on a changefeed with DELETE-related filtering, while the same workload can continue when batch-dml-enable=false

What did you do?

We hit this in a real production scenario (table / changefeed names are sanitized below).

The upstream business workload contains frequent UPDATE / DELETE / INSERT operations on the same business key. The table has a primary key plus a unique key on business columns.

A simplified sanitized table schema is:

CREATE TABLE masked_db.masked_table (
  id BIGINT PRIMARY KEY,
  biz_key VARCHAR(64) NOT NULL,
  biz_status TINYINT NOT NULL,
  payload VARCHAR(64) DEFAULT NULL,
  UNIQUE KEY uk_biz (biz_key, biz_status)
);

We observed different behavior between two changefeeds that are identical except for batch-dml-enable.

Changefeed A (problematic)
DELETE-related filtering is enabled, and the sink keeps the default batch-dml-enable=true.

Sanitized config excerpt:

filter:
  rules:
    - "masked_db.masked_table"
  event-filters:
    - matcher:
        - "masked_db.masked_table"
      ignore-event:
        - delete

Sink URI:

mysql://:@:4000/

the upstream and downstream are both v8.5.4 TiDB cluster

Changefeed B (works)
The same table, same workload, and same DELETE-related filtering, but with batch DML disabled.

Sanitized config excerpt:

filter:
  rules:
    - "masked_db.masked_table"
  event-filters:
    - matcher:
        - "masked_db.masked_table"
      ignore-event:
        - delete

Sink URI:
mysql://:@:4000/?batch-dml-enable=false

the upstream and downstream are both v8.5.4 TiDB cluster

In both cases, the upstream workload is the same. The only intended behavior difference is whether the sink uses batch DML.

And for the same cdc cluster, we have another changefeed which also include the table but with no ignore-event at all and it also works. But the panic in the problematic changefeed affect the normal one.

What did you expect to see?

We expected both changefeeds to behave consistently for the same logical workload.

More specifically:

If DELETE-related filtering changes the event composition that reaches the sink, TiCDC should still handle the resulting event sequence safely.
Even if batch-dml-enable=true causes the sink to detect an invalid same-key combination during batch legality check, TiCDC should not panic.
Instead, it should fallback to a safer path, for example:
non-batch / sequential execution, or
the existing safer retry behavior
At minimum, this should be contained to the changefeed / sink path and should not escalate to a TiCDC process-level panic / restart.

What did you see instead?

We observed different runtime behavior depending on the batch-dml-enable switch:

With Changefeed A (batch-dml-enable=true), TiCDC may panic in the MySQL sink when the filtered workload forms certain same-key combinations.
With Changefeed B (batch-dml-enable=false), the same logical workload can continue replicating, because it avoids the batch merge legality-check path and can proceed through a safer non-batch path.
From the observed behavior, the problematic same-key combinations are consistent with cases like:

Update(K) -> Insert(K)
or, after retry / safer-path transformation, Delete(K) -> Insert(K) -> Insert(K)
So the effective behavior becomes:

batch-dml-enable=true: may panic
batch-dml-enable=false: can continue for the same workload
This is surprising because batch-dml-enable=true is the default, but it behaves less safely than the non-batch path for this production scenario.

We suspect the root cause is that after DELETE-related filtering changes the event composition seen by the sink, the batch legality check may treat the remaining sequence as invalid, while the non-batch path can still continue.

Versions of the cluster

Upstream TiDB cluster version:

TiDB v8.5.4
Upstream TiKV version:

TiKV v8.5.4
TiCDC version:

TiCDC v8.5.7

Start in the TiCDC MySQL sink's batch-DML legality-check path and compare it with the non-batch path using the DELETE-filtered changefeed configuration described here. Reproduce the same-key UPDATE/INSERT or DELETE/INSERT sequence on the stated TiDB versions; done means the changefeed does not panic or restart and the workload follows a safe execution path.