`tiup cluster start <cluster_name>` output shows success, but no TiKV process runs on the designated nodes
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 466
- Forks
- 338
- Avg merge
- 3d 7h
- Merged PRs (30d)
- 8
Description
Bug Report
Please answer these questions before submitting your issue. Thanks!
- What did you do?
Step 1 - Use the following topology.yaml:
global:
user: "tikv"
ssh_port: 22
deploy_dir: "/tikv-deploy"
data_dir: "/tikv-data"
pd_servers:
- host: 192.168.151.250
- host: 192.168.133.47
- host: 192.168.172.40
tikv_servers:
- host: 192.168.173.212
- host: 192.168.131.50
- host: 192.168.131.101
monitoring_servers:
- host: 192.168.202.125
grafana_servers:
- host: 192.168.202.125
Step 2 - Run tiup cluster check ./topology.yaml --apply --user root and observe that one Fail persists: service irqbalance is not running.
Step 3 - Run tiup cluster deploy tikv v6.5.2 ./topology.yaml --user root, followed by tiup cluster start tikv. Observe the following in the output:
Starting component tikv
Starting instance 192.168.131.101:20160
Starting instance 192.168.173.212:20160
Starting instance 192.168.131.50:20160
Start instance 192.168.131.50:20160 success
Start instance 192.168.173.212:20160 success
Start instance 192.168.131.101:20160 success
- What did you expect to see?
When running tiup cluster display tikv, we expect to see:
192.168.131.101:20160 tikv 192.168.131.101 20160/20180 linux/x86_64 Up /tikv-data/tikv-20160 /tikv-deploy/tikv-20160
192.168.131.50:20160 tikv 192.168.131.50 20160/20180 linux/x86_64 Up /tikv-data/tikv-20160 /tikv-deploy/tikv-20160
192.168.173.212:20160 tikv 192.168.173.212 20160/20180 linux/x86_64 Up /tikv-data/tikv-20160 /tikv-deploy/tikv-20160
and in the nodes themselves, expect to see non-empty output upon running sudo lsof -i:20160.
- What did you see instead?
192.168.131.101:20160 tikv 192.168.131.101 20160/20180 linux/x86_64 Down /tikv-data/tikv-20160 /tikv-deploy/tikv-20160
192.168.131.50:20160 tikv 192.168.131.50 20160/20180 linux/x86_64 Down /tikv-data/tikv-20160 /tikv-deploy/tikv-20160
192.168.173.212:20160 tikv 192.168.173.212 20160/20180 linux/x86_64 Down /tikv-data/tikv-20160 /tikv-deploy/tikv-20160
and in the nodes themselves, an empty output upon running sudo lsof -i:20160.
- What version of TiUP are you using (
tiup --version)?
1.12.1
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the tiup cluster check, deploy, start, and display command flow described in the report, reproducing it with the supplied topology.yaml and TiUP 1.12.1. Compare the start output with tiup cluster display and sudo lsof -i:20160 on each designated node; done means failed starts are detected rather than reported as successful, and process status matches the command output.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- cli, devops, infrastructure
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100