pingcap / pingcap/ticdc

logpuller: remove unnecessary event buffering

Open
#6,021 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

affects-8.5 type/enhancement
Dominant language
Go
Stars
56
Forks
63
Avg merge
2d 20h
Merged PRs (30d)
34

Description

Background

The log puller currently enables buffering in the dynamic stream to avoid cleanup work being triggered directly from event handling.

With span cleanup deferred to the failure handler's run loop, event handling no longer needs to call back into the dynamic stream.

Proposal

Disable the dynamic stream buffer and defer drained-span cleanup to the failure handler.

Benefits

  • Simplify the event-processing path.
  • Remove unnecessary buffering and its associated overhead.
  • Keep span cleanup outside of dynamic stream event handling.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by locating the log puller's dynamic stream and failure handler. Trace how buffering is enabled and where drained-span cleanup is currently triggered, then verify that cleanup is deferred to the failure handler's run loop and no longer occurs during dynamic stream event handling.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
stream-processing
Issue type
Refactor
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.