Error: failed to start tikv
- Dominant language
- Go
- Stars
- 40.5k
- Forks
- 6.2k
- PR merge metrics
- PR metrics pending
Description
## Bug Report
Please answer these questions before submitting your issue. Thanks!
### 1. Minimal reproduce step (Required)
## Bug Report
Please answer these questions before submitting your issue. Thanks!
1. What did you do?
When running `$ tiup cluster start tidb-test`, I received an error
```
Starting component pd
Starting instance node4:2379
Starting instance node3:2379
Starting instance node5:2379
Start instance node5:2379 success
Start instance node3:2379 success
Start instance node4:2379 success
Starting component tikv
Starting instance node8:20162
Starting instance node6:20160
Starting instance node7:20161
Error: failed to start tikv: failed to start: node7 tikv-20161.service, please check the instance's log(/local/tidb-deploy/tikv-20161/log) for more detail.: timed out waiting for port 20161 to be started after 2m0s
Verbose debug logs has been written to /users/nobi/.tiup/logs/tiup-cluster-debug-2022-02-16-02-18-50.log.
```
This is the output of `$ tiup cluster display tidb-test`

And here is the error message in /local/tidb-deploy/tikv-20161/log/tikv.log:
```
[2022/02/16 02:23:18.076 -07:00] [WARN] [server.rs:435] ["failed to remove space holder on starting: No such file or directory (os error 2)"]
[2022/02/16 02:23:20.938 -07:00] [FATAL] [lib.rs:465] ["Failed to reserve space for recovery: Operation not supported (os error 95)."] [backtrace=" 0: tikv_util::set_panic_hook::{{closure}}\n at /home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tikv/components/tikv_util/src/lib.rs:464:18\n 1: std::panicking::rust_panic_with_hook\n at /rustc/2faabf579323f5252329264cc53ba9ff803429a3/library/std/src/panicking.rs:626:17\n 2: std::panicking::begin_panic_handler::{{closure}}\n at /rustc/2faabf579323f5252329264cc53ba9ff803429a3/library/std/src/panicking.rs:519:13\n 3: std::sys_common::backtrace::__rust_end_short_backtrace\n at /rustc/2faabf579323f5252329264cc53ba9ff803429a3/library/std/src/sys_common/backtrace.rs:141:18\n 4: rust_begin_unwind\n at /rustc/2faabf579323f5252329264cc53ba9ff803429a3/library/std/src/panicking.rs:515:5\n 5: std::panicking::begin_panic_fmt\n at /rustc/2faabf579323f5252329264cc53ba9ff803429a3/library/std/src/panicking.rs:457:5\n 6: server::server::TiKVServer::init_fs::{{closure}}\n at /home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tikv/components/server/src/server.rs:445:26\n 7: core::result::Result::map_err\n at /rustc/2faabf579323f5252329264cc53ba9ff803429a3/library/core/src/result.rs:835:27\n server::server::TiKVServer::init_fs\n at /home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tikv/components/server/src/server.rs:441:13\n server::server::run_tikv\n at /home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tikv/components/server/src/server.rs:155:9\n 8: tikv_server::main\n at /home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tikv/cmd/tikv-server/src/main.rs:190:5\n 9: core::ops::function::FnOnce::call_once\n at /rustc/2faabf579323f5252329264cc53ba9ff803429a3/library/core/src/ops/function.rs:227:5\n std::sys_common::backtrace::__rust_begin_short_backtrace\n at /rustc/2faabf579323f5252329264cc53ba9ff803429a3/library/std/src/sys_common/backtrace.rs:125:18\n 10: main\n 11: __libc_start_main\n at /build/glibc-2ORdQG/glibc-2.27/csu/../csu/libc-start.c:310\n 12: \n"] [location=components/server/src/server.rs:445] [thread_name=main]
```
4. What version of TiUP are you using (`tiup --version`)?
tiup version: 1.9.0
tidb version: 5.4.0
The cluster is set up on Emulab, and here is the config:
https://github.com/20211202na/tidb-data/blob/main/topology.yaml
Contributor guide
Assessment
This issue has not been assessed yet.