pingcap / pingcap/tidb-operator

Adding Pod restart window config

Open
#4,822 1 comment 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Go
Stars
1.3k
Forks
540
Avg merge
3d 2h
Merged PRs (30d)
18

Description

Feature Request

Is your feature request related to a problem? Please describe:

Currently tidb-operator supports maintenance window via updating the pause flag in TidbCluster CR(doc). In our case we have a TiDB cluster that spans across multiple k8s cluster(i.e., deployment managed by multiple operator), and we want to ensure Pod restarts by pod controller (triggered by annotation) in different clusters don't overlap. It's possible to achieve this by turning on/off the pause flag, but we ant to avoid constantly updating the TidbCluster CR.

Describe the feature you'd like:

  1. Adding a config in tidb-operator that defines a maintenance window where all reconciles are paused. For instance we can use a CRON schedule+duration to define the window. The following example configures an operation window from 0-8 every day.
    --maintenance-schedule="0 0 * * *"
    --maintenance-duration="8h"
  1. The pod controller should skip Pod restarts if it's not in operation window. For example, in the sync loop of pod controller
func (c *PodController) sync(key string) (reconcile.Result, error) {
	if c.inOperationWindow(time.Now().UTC()) {
		return reconcile.Result{}, nil
	}
...

Describe alternatives you've considered:

Teachability, Documentation, Adoption, Migration Strategy:

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the pause-flag behavior described in the linked maintenance-window documentation and the sync entry point in pkg/controller/tidbcluster/pod_control.go around line 153. Determine how a cron schedule and duration should define the operation window and how that affects pod-controller restarts; done means the configured window prevents the requested reconciles or restarts without repeatedly updating the TidbCluster CR.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, kubernetes
Domain
infrastructure
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.