pingcap / pingcap/tiup

should remove tidb /info key in PD(etcd) after scale-in TiDB

Open
#1,752 2 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

type/bug
Dominant language
Go
Stars
466
Forks
338
Avg merge
3d 7h
Merged PRs (30d)
8

Description

Bug Report

Please answer these questions before submitting your issue. Thanks!

  1. What did you do?

topo.yaml

pd_servers:
  - host: 10.0.1.9

tikv_servers:
  - host: 10.0.1.9

tidb_servers:
  - host: 10.0.1.9
  1. Deploy a tidb cluster.
tiup cluster deploy dev nightly topo.yaml 
tiup cluster start dev
  1. After deploy tidb cluster, you can use etcdctl to find the topology information in PD(etcd).
▶ etcdctl --endpoints=10.0.1.9:2379 get "/topology/tidb" --prefix
/topology/tidb/10.0.1.9:4000/info
{"version":"v5.5.0-alpha","git_hash":"adcaaa5cbe1058da8f363c173521929405d0ecb6","ip":"10.0.1.9","status_port":10080,"deploy_path":"/home/tidb/deploy/tidb-4000/bin","start_timestamp":1644481689,"labels":{}}
/topology/tidb/10.0.1.9:4000/ttl
1644481749835274163

You can see there are 2 keys in PD(etcd), those 2 keys are written into PD(etcd) by TiDB itself.

  • /topology/tidb/10.0.1.9:4000/info
  • /topology/tidb/10.0.1.9:4000/ttl
  1. Now, scale-in tidb.
tiup cluster scale-in dev -N 10.0.1.9:4000
  1. Now, use etcdctl to find the topology information in PD(etcd) again:
▶ etcdctl --endpoints=10.0.1.9:2379 get "/topology/tidb" --prefix
/topology/tidb/10.0.1.9:4000/info
{"version":"v5.5.0-alpha","git_hash":"adcaaa5cbe1058da8f363c173521929405d0ecb6","ip":"10.0.1.9","status_port":10080,"deploy_path":"/home/tidb/deploy/tidb-4000/bin","start_timestamp":1644481689,"labels":{}}

As you can see, /topology/tidb/10.0.1.9:4000/info is still in PD(etcd), it's unexpected.
/topology/tidb/10.0.1.9:4000/ttl key has been auto delete after ttl timeout.

/topology/tidb/10.0.1.9:4000/info should be deleted after TiDB is been scale-in.

What I expected?

TiUP should remove following keys in PD(etcd) when tidb and ngm were been scale-in.

  • /topology/xxx/10.0.1.9:4000/info
  • /topology/xxx/10.0.1.9:4000/ttl
Other consideration

Since /topology/tidb/10.0.1.9:4000/info and /topology/tidb/10.0.1.9:4000/ttl are written into PD(etcd) by TiDB itself, so let TiUP to delete those key maybe look weird. Maybe let TiUP to write and delte those keys is better.

  1. What version of TiUP are you using (tiup --version)?
▶ tiup --version
1.8.2 tiup
Go Version: go1.17.5
Git Ref: v1.8.2
GitHash: a2912cf0470d651ce724e6c4e79871ec772abe38

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No file or test is named. Start by tracing the tiup cluster scale-in command and its handling of PD(etcd) topology keys, then compare cleanup of the /topology/tidb/.../info and /ttl entries. Done means scale-in removes the expected TiDB and ngm keys, with tests covering the cleanup behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
cli, distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.