pingcap / pingcap/ticdc

dispatcher orchestrator: maintainer epoch mismatch should not be log-only

Open
#4,254 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Go
Stars
56
Forks
63
Avg merge
2d 20h
Merged PRs (30d)
34

Description

Problem

In dispatcher orchestrator bootstrap path, maintainer epoch mismatch is only logged and does not trigger any corrective handling.

Current behavior (approx):

  • compare manager.GetMaintainerEpoch() with cfConfig.Epoch
  • on mismatch, only log error: "maintainer epoch changed, this should not happen"

Why this matters

This is a control-plane consistency risk and is not specific to recover flow.
If epoch mismatch indicates stale lifecycle state, only logging may leave the system in ambiguous state and make switch-window issues harder to converge and diagnose.

Current code references

  • downstreamadapter/dispatcherorchestrator/dispatcher_orchestrator.go (bootstrap request handling)
  • mismatch check around the temporary FIXME section (manager.GetMaintainerEpoch() != cfConfig.Epoch)

Expected behavior

When epoch mismatch is detected, we should have explicit handling, at least one of:

  • reject/ignore stale control request paths tied to old epoch,
  • trigger deterministic re-bootstrap / reconciliation,
  • or surface a stronger signal (metrics/alert) with bounded retry strategy.

Suggested direction

  1. Define clear epoch-mismatch semantics (what is stale, what is authoritative).
  2. Implement a deterministic action instead of log-only behavior.
  3. Add observability beyond logs (counter/alert label by changefeed).
  4. Add focused tests for mismatch during maintainer switch windows.

Scope

Cross-feature control-plane behavior (bootstrap/schedule/control messages), not limited to recover request handling.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Read downstreamadapter/dispatcherorchestrator/dispatcher_orchestrator.go, starting at the bootstrap request handling and the temporary FIXME around manager.GetMaintainerEpoch() != cfConfig.Epoch. Define which epoch is authoritative and which control paths are stale, then determine the deterministic handling and observability needed. Done means focused tests cover mismatch during maintainer switch windows and log-only behavior is replaced by the chosen bounded action.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
backend, distributed-systems
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.