pingcap / pingcap/ticdc

dynstream: simplify AddPath API and remove variadic AreaSettings usage

Open
#4,288 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Go
Stars
56
Forks
63
Avg merge
2d 20h
Merged PRs (30d)
34

Description

Background

utils/dynstream currently exposes:

AddPath(path P, dest D, area ...AreaSettings) error

After introducing path-level batching (count / bytes), callers now need to build and pass AreaSettings as an optional argument in AddPath. In practice, this has increased call-site complexity and introduced maintainability concerns.

Problem Statement

  1. Unclear API semantics
    AddPath appears to be a plain path-registration API, but it also carries optional configuration injection.

  2. Easy-to-misuse variadic parameter
    The implementation only reads area[0]. Extra arguments are silently ignored, with no explicit validation or error.

  3. Mixed responsibilities in configuration model
    AreaSettings currently mixes area-level memory-control settings and path-level batching settings.

  4. Higher caller cognitive load
    Different modules (event collector / log puller / heartbeat collector) need to construct a relatively heavy settings object even for simple scenarios.

Impact

  • Reduced API readability and maintainability.
  • Higher chance of “works but semantically inconsistent” call-site behavior.
  • Future feature evolution (especially path-level settings) is harder due to coupling.

Desired Goals

  • Separate semantics between “basic path registration” and “path registration with settings”.
  • Keep behavior backward-compatible while reducing misuse risk.
  • Lower integration and maintenance costs for call sites.

Proposed Direction

Option A (recommended, minimal and practical)
  • Keep: AddPath(path, dest)
  • Add: AddPathWithSettings(path, dest, settings AreaSettings)
  • Internally share one implementation and remove variadic settings usage.
Option B (further decoupling)
  • Split settings models:
    • AreaSettings: area-level memory-control only
    • PathSettings / PathBatchSettings: path-level batching only
  • Let AddPathWithSettings accept explicit settings structures to avoid cross-layer coupling.

Acceptance Criteria

  • API no longer relies on variadic AreaSettings in AddPath.
  • All existing call sites are migrated with behavior preserved.
  • Add focused unit tests for:
    • default path registration
    • registration with settings
    • invalid/misuse branches

Notes

This is not blocking current feature delivery, but it should be scheduled soon to prevent higher refactor cost in future changes.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in utils/dynstream by tracing AddPath and its AreaSettings handling, then inspect the event collector, log puller, and heartbeat collector call sites. Migrate registrations while preserving behavior, remove variadic usage, and add focused tests for default registration, settings-based registration, and invalid or misuse branches.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
backend
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.