pingcap / pingcap/tidb-operator

Instead of creating the resource in every sync cycle, first check whether it already exists.

Open
#6,322 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Go
Stars
1.3k
Forks
540
Avg merge
3d 2h
Merged PRs (30d)
18

Description

Bug Report

What version of Kubernetes are you using?

EKS v1.30

What version of TiDB Operator are you using?

all version

What storage classes exist in the Kubernetes cluster and what are used for PD/TiKV pods?

What's the status of the TiDB cluster pods?

What did you do?

What did you expect to see?
Instead of creating the resource in every sync cycle, first check whether it already exists.

What did you see instead?
TiDB Operator uses a reconcile mechanism. In each sync cycle, it attempts to create resources. Since the resources already exist and haven't changed, Kubernetes returns a 409 Conflict error. As shown in the picture, you can see many create attempts followed by 409 errors.
By default, TiDB Operator reconciles every 30 seconds. If the TidbCluster (TC) status changes frequently—for example, under a high workload where tc.status.tikv.leaderCount is constantly changing—reconciliation will also be triggered more frequently.

Image

Due to the high frequency of reconciliations, Kubernetes throttling may occur.

I0724 03:28:44.147312 1 request.go:645] Throttling request took 1.087973587s, request: POST:https://10.100.0.1:443/api/v1/namespaces/tidb-cluster/configmaps

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing the TiDB Operator reconcile cycle and its resource creation calls, focusing on the ConfigMap POST requests described in the report. Determine how existing resources are handled and verify that repeated reconciliations no longer produce 409 Conflict errors or unnecessary create attempts.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, kubernetes
Domain
infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.