pingcap / pingcap/ticdc

[P1][Logs] Aggregate region anomaly logs instead of logging per object

Open
#4,737 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

type/enhancement
Dominant language
Go
Stars
56
Forks
63
Avg merge
2d 20h
Merged PRs (30d)
34

Description

Parent issue: #4691

Intent

Region-related anomalies such as slow regions, holes, stale requests, and untracked regions are currently prone to per-object logging. Under large region counts, the log volume grows with the number of affected regions instead of with the number of distinct incidents. This task moves those logs to aggregated summaries with small samples.

Modification points

  • Replace per-region anomaly logs with window-based summaries plus top-N samples.
  • Keep aggregate fields such as count, maxAge, largestGap, sampleRegionIDs, and sampleSubscriptionIDs.
  • Reserve full per-region detail for debug or diagnostics-only paths.

Entry points

  • logservice/logpuller/subscription_client.go:1055
  • logservice/logpuller/subscription_client.go:1061
  • logservice/logpuller/subscription_client.go:1065
  • logservice/logpuller/subscription_client.go:1070
  • logservice/logpuller/region_request_worker.go:280
  • logservice/logpuller/region_request_worker.go:334
  • logservice/logpuller/region_req_cache.go:166
  • logservice/logpuller/region_req_cache.go:247

Done criteria

  • Region anomaly logs no longer scale linearly with the number of affected regions on the default log path.
  • The summary logs still expose the size, severity, and representative samples of the incident.
  • Full detail is available only when explicitly requested through a debug or diagnostic path.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the listed entry points in subscription_client.go, region_request_worker.go, and region_req_cache.go, tracing how region anomalies are currently logged. Check how the affected regions and subscriptions are collected, then verify that default logs use window summaries with counts, severity fields, and top-N samples while full detail remains limited to debug or diagnostic paths.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
observability
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.