pingcap / pingcap/tiup

can't prune cluster due to false meta_yaml

Open
#2,004 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

type/bug
Dominant language
Go
Stars
466
Forks
338
Avg merge
3d 7h
Merged PRs (30d)
8

Description

Bug Report

Please answer these questions before submitting your issue. Thanks!

  1. What did you do?

I scale-in my cluster, then want to prune data, but some nodes don't appear in the prune list

tiup cluster display result

xxx.xxx.166.2:20080    tikv          xxx.xxx.166.2    20080/20081  linux/x86_64  Tombstone        /mnt/disk/8/xxx/tikv/20080       /home/kvstore/xxx/tikv-20080
xxx.xxx.166.2:20090    tikv          xxx.xxx.166.2    20090/20091  linux/x86_64  Tombstone        /mnt/disk/9/xxx/tikv/20090       /home/kvstore/xxx/tikv-20090

but the prune list is empty

Will destroy these nodes: []

This is because meta_yaml is not showing the right information.
now meta_yaml

- host: xxx.xxx.166.2
ssh_port: 22
port: 20080
status_port: 20081
deploy_dir: /home/kvstore/xxx/tikv-20080
data_dir: /mnt/disk/8/xxx/tikv/20080
log_dir: /home/kvstore/xxx/tikv-20080/log
config:
  server.labels:
    host: xxx.xxx.166.2
arch: amd64
os: linux

expect meta_yaml

- host: xxx.xxx.166.2
ssh_port: 22
port: 20080
status_port: 20081
deploy_dir: /home/kvstore/xxx/tikv-20080
data_dir: /mnt/disk/8/xxx/tikv/20080
log_dir: /home/kvstore/xxx/tikv-20080/log
offline: true.  // offline not set now
config:
  server.labels:
    host: xxx.xxx.166.2
arch: amd64
os: linux

maybe we should provide an interface to refresh meta_yaml?

  1. What version of TiUP are you using (tiup --version)?
    v1.10.2

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the scale-in, tiup cluster display, and prune flow described in the report, then trace how meta.yaml is generated and how the prune command reads it. Done means the metadata records the offline state for tombstone nodes and the prune list includes those nodes; no source files or tests are named in the issue.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
cli
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.