pingcap / pingcap/ticdc

[P3][Logs] Replace per-dispatcher lifecycle logs with manager and changefeed summaries

Open
#4,752 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

type/enhancement
Dominant language
Go
Stars
56
Forks
63
Avg merge
2d 20h
Merged PRs (30d)
34

Description

Parent issue: #4691

Intent

Dispatcher lifecycle logs are currently emitted per object for events such as add, commit, ready, remove, drop, and handshake. When the number of dispatchers is high, the resulting logs are large in volume but low in marginal information. This task shifts lifecycle visibility to manager-level and changefeed-level summaries.

Modification points

  • Replace per-dispatcher lifecycle narration with periodic or window-based summaries at the manager or changefeed layer.
  • Keep counts for key lifecycle events such as added, removed, reset, handshake, and drop.
  • Reserve per-dispatcher details for abnormal cases or diagnostics mode.

Entry points

  • downstreamadapter/eventcollector/event_collector.go:259
  • downstreamadapter/eventcollector/event_collector.go:287
  • downstreamadapter/eventcollector/event_collector.go:302
  • downstreamadapter/eventcollector/event_collector.go:329
  • downstreamadapter/eventcollector/dispatcher_stat.go:603
  • downstreamadapter/eventcollector/dispatcher_stat.go:621
  • downstreamadapter/eventcollector/dispatcher_stat.go:659
  • downstreamadapter/eventcollector/dispatcher_stat.go:670
  • logservice/logpuller/region_event_handler.go:387

Done criteria

  • Large dispatcher create and delete waves produce summary-oriented logs by default.
  • Managers expose the important counts without logging each dispatcher lifecycle step.
  • Detailed per-dispatcher logs remain available only for exceptions or targeted diagnostics.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading the listed entry points in downstreamadapter/eventcollector/event_collector.go and dispatcher_stat.go, then inspect logservice/logpuller/region_event_handler.go. Trace where dispatcher lifecycle events are logged and how manager or changefeed context is available. Done means large create/delete waves produce summary logs by default, with per-dispatcher details limited to exceptions or diagnostics.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
observability
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.