pingcap / pingcap/ticdc

[Bug] Reduce excessive repeated log when memory quota is not set

Open
#4,890 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Go
Stars
56
Forks
63
Avg merge
2d 20h
Merged PRs (30d)
34

Description

Description

The following log message is emitted too frequently:

available memory quota is not set, skip scan

In one observed case, this single message produced nearly 2,000,000 lines.

Impact

  • Significant log volume and storage overhead
  • Important logs are drowned out by repeated noise
  • Potential extra I/O overhead from excessive logging

Expected Behavior

When memory quota is not set, this message should be rate-limited, deduplicated, or logged once per component/time window.

Actual Behavior

The same message is repeatedly printed at very high frequency (about 2 million lines in one case).

Suggested Fix

  • Add rate limiting or sampling for this log
  • Or log this condition once and suppress repetitive duplicates
  • Optionally emit periodic summary counters instead of per-event logging

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Search the repository for the exact message "available memory quota is not set, skip scan" and trace the repeated logging path. Read the surrounding scan or memory-quota handling and its existing tests, then verify that an unset quota no longer produces unbounded duplicate logs while the condition remains observable.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
observability
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.