pingcap / pingcap/ticdc

ClassifyEvent() does not recognize DROP FOREIGN KEY causing CheckpointTS stuck

Open
#6,275 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

type/bug
Dominant language
Go
Stars
56
Forks
63
Avg merge
2d 20h
Merged PRs (30d)
34

Description

What did you do?
  1. Start a cluster

    tiup playground-ng v8.5.8 --db 1 --kv 1 --pd 1 --tiflash 0 --ticdc 1 --ticdc.config ./ticdc-newarch.toml
    
  2. Prepare changefeed config, containing at least one arbitrary event filter

    [filter]
    [[filter.event-filters]]
    matcher = ['*.*']
    ignore-event = ['delete']
    
  3. Create a changefeed

    tiup cdc:v8.5.8 cli changefeed create -c test-8732 --sink-uri 'blackhole://' --config ./config-8732.toml
    
  4. Create a table with foreign key

    use test;
    -- set global tidb_enable_dist_task = 0;
    -- set global tidb_ddl_enable_fast_reorg = 0;
    create table b (id bigint primary key);
    create table a (id bigint primary key, b bigint, foreign key fk_1 (b) references b (id));
    
  5. Drop the foreign key

    alter table a drop foreign key fk_1;
    
What did you expect to see?

No response

What did you see instead?
  • CheckpointTS is now stuck.
  • CDC log keeps showing errors logs like this every second:
[ERROR] [event_broker.go:726] ["scan events failed"] [changefeedID=default/test-8732] [dispatcherID=62170044884923103651170238435157499028] [tableID=134] [dataRange="{\"ClusterID\":0,\"Span\":{\"TableID\":134,\"StartKey\":\"dIAAAAAAAAD/hl9yAAAAAAD6\",\"EndKey\":\"dIAAAAAAAAD/hl9zAAAAAAD6\"},\"CommitTsStart\":469142236153511947,\"CommitTsEnd\":469142250899111940,\"LastScannedTxnStartTs\":0}"] [receivedResolvedTs=469142251162304517] [sentResolvedTs=469142236153511947] [error="event type drop foreign key not valid"] 

Restarting changefeed cannot fix the lag.

Versions of the cluster

v8.5.8

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the issue with the provided TiUP, changefeed, table-creation, and DROP FOREIGN KEY commands, then inspect ClassifyEvent() and the CDC error showing "event type drop foreign key not valid." No source file or test is named in the report. Done means DROP FOREIGN KEY is recognized and the changefeed's CheckpointTS no longer remains stuck.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, sql
Domain
backend, databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.