pingcap / pingcap/tiflow

CDC: ErrMessageTooLarge does not take compression into account

Open
#12,232 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

affects-7.5 area/ticdc may-affects-6.5 may-affects-7.1 may-affects-8.1 may-affects-8.5 severity/major type/bug
Dominant language
Go
Stars
464
Forks
313
Avg merge
4d 7h
Merged PRs (30d)
14

Description

What did you do?
  1. Prepare a Kafka server at 10.0.0.9:9092
  2. Create 1 topic:
    • topic_small with max.message.bytes = 100000 (100KB)
  3. Create a changefeed with
    • sink-uri = kafka://10.0.0.9:9092/topic_small?protocol=canal-json&compression=gzip
  4. Execute the following upstream:
    create table test.big (pk bigint primary key, content longtext not null);
    insert into test.big values (2, repeat('y', 100000));
    
What did you expect to see?

Because of compression the actual message being sent will be well below 100 KB, so Kafka should accept the message, and so should TiCDC.

What did you see instead?

TiCDC actually blocks the message saying "Single message is too large for canal-json", citing length being well above 100 KB.

Versions of the cluster

TiCDC version (execute cdc version):

v7.5.6

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the reported Kafka changefeed scenario with canal-json and gzip, then trace the ErrMessageTooLarge check and the compression handling it compares against. Done means a 100 KB upstream row can be sent when its compressed Kafka message is within the topic limit, while genuinely oversized messages are still rejected; add or update a regression test if the existing test structure covers this path.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, kafka
Domain
data-engineering, stream-processing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.