pingcap / pingcap/ticdc

[P4][Logs] Replace large zap.Any payloads on hot paths with compact fields

Open
#4,747 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

type/enhancement
Dominant language
Go
Stars
56
Forks
63
Avg merge
2d 20h
Merged PRs (30d)
34

Description

Parent issue: #4691

Intent

Some hot-path logs include large objects, arrays, or raw payloads through zap.Any. Those logs are expensive to encode, expensive to store, and still hard to read because they lack a concise summary. This task replaces large payload dumps on default paths with compact structured fields.

Modification points

  • Replace large zap.Any fields on Info and Warn logs with scalar fields and small samples.
  • Keep raw events, config blobs, or object dumps only behind Debug or a dedicated diagnostics mode.
  • Ensure the retained fields answer the first triage question directly instead of forcing the reader to inspect a large payload.

Entry points

  • downstreamadapter/dispatcherorchestrator/dispatcher_orchestrator.go:162
  • downstreamadapter/eventcollector/dispatcher_stat.go:521
  • downstreamadapter/eventcollector/dispatcher_stat.go:670
  • logservice/txnutil/lock_resolver.go:56
  • logservice/logpuller/subscription_client.go:1012
  • logservice/logpuller/subscription_client.go:1057
  • logservice/logpuller/subscription_client.go:1072
  • logservice/logpuller/region_request_worker.go:298

Done criteria

  • Default logs no longer include large arrays, raw event objects, or config blobs on the targeted hot paths.
  • The remaining fields make the incident understandable without inspecting a full dump.
  • Full payload detail is still available through lower-level or diagnostics-only logging.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Review the listed call sites in dispatcher_orchestrator.go, dispatcher_stat.go, lock_resolver.go, subscription_client.go, and region_request_worker.go, starting with the specified log statements and their hot-path data. Identify which Info and Warn fields use large zap.Any payloads, then verify that default logs retain concise triage fields while full payloads remain available only through lower-level or diagnostics-only logging.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
observability
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.