[Bug] 2022-12-14 02:03:30,954 [main] WARN o.a.i.d.c.ConfigNodeClient:254 - The current node may have been down TEndPoint(ip:192.168.57.10, port:22277),try next node
- Dominant language
- Java
- Stars
- 6.4k
- Forks
- 1.2k
- Avg merge
- 1d 23h
- Merged PRs (30d)
- 115
Description
### Search before asking
- [X] I searched in the [issues](https://github.com/apache/iotdb/issues) and found nothing similar.
### Version
1.0
### Describe the bug and provide the minimal reproduce step
all parameters are
---------------------
Starting IoTDB DataNode
---------------------
setting local JMX...
Maximum memory allocation pool = 914MB, initial memory allocation pool = 200MB
If you want to change this configuration, please check conf/datanode-env.sh.
2022-12-14 02:03:30,796 [main] INFO o.a.i.d.c.IoTDBDescriptor:161 - Start to read config file file:./sbin/../conf/iotdb-common.properties
2022-12-14 02:03:30,802 [main] INFO o.a.i.d.c.IoTDBDescriptor:178 - Start to read config file file:./sbin/../conf/iotdb-datanode.properties
2022-12-14 02:03:30,822 [main] INFO o.a.i.d.c.IoTDBDescriptor:1582 - initial allocateMemoryForRead = 255590400
2022-12-14 02:03:30,823 [main] INFO o.a.i.d.c.IoTDBDescriptor:1583 - initial allocateMemoryForWrite = 255590400
2022-12-14 02:03:30,823 [main] INFO o.a.i.d.c.IoTDBDescriptor:1584 - initial allocateMemoryForSchema = 85196800
2022-12-14 02:03:30,823 [main] INFO o.a.i.d.c.IoTDBDescriptor:1585 - initial allocateMemoryForConsensus = 85196800
2022-12-14 02:03:30,823 [main] INFO o.a.i.d.c.IoTDBDescriptor:1718 - allocateMemoryForSchemaRegion = 68157440
2022-12-14 02:03:30,824 [main] INFO o.a.i.d.c.IoTDBDescriptor:1721 - allocateMemoryForSchemaCache = 8519680
2022-12-14 02:03:30,824 [main] INFO o.a.i.d.c.IoTDBDescriptor:1725 - allocateMemoryForPartitionCache = 0
2022-12-14 02:03:30,824 [main] INFO o.a.i.d.c.IoTDBDescriptor:1728 - allocateMemoryForLastCache = 8519680
2022-12-14 02:03:30,843 [main] INFO o.a.i.t.c.c.TSFileDescriptor:129 - try loading iotdb-common.properties from /app/iotdb/apache-iotdb-1.0.0-all-bin/./sbin/../conf/iotdb-common.properties
2022-12-14 02:03:30,849 [main] INFO o.a.i.d.c.IoTDBDescriptor:382 - IoTDB enable memory control: true
2022-12-14 02:03:30,914 [main] INFO o.a.i.c.c.IoTDBThreadPoolFactory:186 - new single scheduled thread pool: Stateful-Trigger-Information-Updater
2022-12-14 02:03:30,916 [main] INFO o.a.i.d.s.DataNodeServerCommandLine:77 - Running mode -s
2022-12-14 02:03:30,928 [main] INFO o.a.i.d.c.IoTDBStartCheck:155 - Starting IoTDB 1.0.0 (Build: fbbca3f-dev)
2022-12-14 02:03:30,937 [main] INFO o.a.i.c.s.StartupChecks:59 - Start JMX locally.
2022-12-14 02:03:30,938 [main] INFO o.a.i.c.s.StartupChecks:43 - JDK version is 8.
2022-12-14 02:03:30,942 [main] INFO o.a.i.d.c.ConfigNodeInfo:94 - Successfully update ConfigNode: [TEndPoint(ip:192.168.57.10, port:22277)].
2022-12-14 02:03:30,954 [main] WARN o.a.i.d.c.ConfigNodeClient:254 - The current node may have been down TEndPoint(ip:192.168.57.10, port:22277),try next node
2022-12-14 02:03:30,954 [main] WARN o.a.i.d.c.ConfigNodeClient:254 - The current node may have been down TEndPoint(ip:192.168.57.10, port:22277),try next node
2022-12-14 02:03:30,955 [main] WARN o.a.i.db.service.DataNode:280 - Cannot register to the cluster, because: Fail to connect to any config node. Please check server status
2022-12-14 02:03:35,964 [main] WARN o.a.i.d.c.ConfigNodeClient:254 - The current node may have been down TEndPoint(ip:192.168.57.10, port:22277),try next node
2022-12-14 02:03:35,966 [main] WARN o.a.i.d.c.ConfigNodeClient:254 - The current node may have been down TEndPoint(ip:192.168.57.10, port:22277),try next node
2022-12-14 02:03:35,967 [main] WARN o.a.i.db.service.DataNode:280 - Cannot register to the cluster, because: Fail to connect to any config node. Please check server status
2022-12-14 02:03:40,971 [main] WARN o.a.i.d.c.ConfigNodeClient:254 - The current node may have been down TEndPoint(ip:192.168.57.10, port:22277),try next node
2022-12-14 02:03:40,973 [main] WARN o.a.i.d.c.ConfigNodeClient:254 - The current node may have been down TEndPoint(ip:192.168.57.10, port:22277),try next node
2022-12-14 02:03:40,974 [main] WARN o.a.i.db.service.DataNode:280 - Cannot register to the cluster, because: Fail to connect to any config node. Please check server status
2022-12-14 02:03:45,987 [main] WARN o.a.i.d.c.ConfigNodeClient:254 - The current node may have been down TEndPoint(ip:192.168.57.10, port:22277),try next node
2022-12-14 02:03:45,989 [main] WARN o.a.i.d.c.ConfigNodeClient:254 - The current node may have been down TEndPoint(ip:192.168.57.10, port:22277),try next node
2022-12-14 02:03:45,990 [main] WARN o.a.i.db.service.DataNode:280 - Cannot register to the cluster, because: Fail to connect to any config node. Please check server status
2022-12-14 02:03:51,003 [main] WARN o.a.i.d.c.ConfigNodeClient:254 - The current node may have been down TEndPoint(ip:192.168.57.10, port:22277),try next node
2022-12-14 02:03:51,005 [main] WARN o.a.i.d.c.ConfigNodeClient:254 - The current node may have been down TEndPoint(ip:192.168.57.10, port:22277),try next node
2022-12-14 02:03:51,006 [main] WARN o.a.i.db.service.DataNode:280 - Cannot register to the cluster, because: Fail to connect to any config node. Please check server status
2022-12-14 02:03:56,013 [main] WARN o.a.i.d.c.ConfigNodeClient:254 - The current node may have been down TEndPoint(ip:192.168.57.10, port:22277),try next node
2022-12-14 02:03:56,014 [main] WARN o.a.i.d.c.ConfigNodeClient:254 - The current node may have been down TEndPoint(ip:192.168.57.10, port:22277),try next node
2022-12-14 02:03:56,014 [main] WARN o.a.i.db.service.DataNode:280 - Cannot register to the cluster, because: Fail to connect to any config node. Please check server status
2022-12-14 02:04:01,035 [main] WARN o.a.i.d.c.ConfigNodeClient:254 - The current node may have been down TEndPoint(ip:192.168.57.10, port:22277),try next node
2022-12-14 02:04:01,037 [main] WARN o.a.i.d.c.ConfigNodeClient:254 - The current node may have been down TEndPoint(ip:192.168.57.10, port:22277),try next node
2022-12-14 02:04:01,038 [main] WARN o.a.i.db.service.DataNode:280 - Cannot register to the cluster, because: Fail to connect to any config node. Please check server status
2022-12-14 02:04:06,047 [main] WARN o.a.i.d.c.ConfigNodeClient:254 - The current node may have been down TEndPoint(ip:192.168.57.10, port:22277),try next node
2022-12-14 02:04:06,049 [main] WARN o.a.i.d.c.ConfigNodeClient:254 - The current node may have been down TEndPoint(ip:192.168.57.10, port:22277),try next node
2022-12-14 02:04:06,050 [main] WARN o.a.i.db.service.DataNode:280 - Cannot register to the cluster, because: Fail to connect to any config node. Please check server status
2022-12-14 02:04:11,061 [main] WARN o.a.i.d.c.ConfigNodeClient:254 - The current node may have been down TEndPoint(ip:192.168.57.10, port:22277),try next node
2022-12-14 02:04:11,064 [main] WARN o.a.i.d.c.ConfigNodeClient:254 - The current node may have been down TEndPoint(ip:192.168.57.10, port:22277),try next node
2022-12-14 02:04:11,064 [main] WARN o.a.i.db.service.DataNode:280 - Cannot register to the cluster, because: Fail to connect to any config node. Please check server status
2022-12-14 02:04:16,080 [main] WARN o.a.i.d.c.ConfigNodeClient:254 - The current node may have been down TEndPoint(ip:192.168.57.10, port:22277),try next node
2022-12-14 02:04:16,082 [main] WARN o.a.i.d.c.ConfigNodeClient:254 - The current node may have been down TEndPoint(ip:192.168.57.10, port:22277),try next node
2022-12-14 02:04:16,083 [main] WARN o.a.i.db.service.DataNode:280 - Cannot register to the cluster, because: Fail to connect to any config node. Please check server status
2022-12-14 02:04:21,094 [main] ERROR o.a.i.db.service.DataNode:297 - Cannot register to the cluster after 10 retries
2022-12-14 02:04:21,126 [main] ERROR o.a.i.db.service.DataNode:181 - Fail to start server
org.apache.iotdb.commons.exception.StartupException: Cannot register to the cluster.
at org.apache.iotdb.db.service.DataNode.registerInConfigNode(DataNode.java:298)
at org.apache.iotdb.db.service.DataNode.doAddNode(DataNode.java:160)
at org.apache.iotdb.db.service.DataNodeServerCommandLine.run(DataNodeServerCommandLine.java:95)
at org.apache.iotdb.commons.ServerCommandLine.doMain(ServerCommandLine.java:58)
at org.apache.iotdb.db.service.DataNode.main(DataNode.java:132)
2022-12-14 02:04:21,127 [main] INFO o.a.i.db.service.DataNode:698 - Deactivating IoTDB DataNode...
2022-12-14 02:04:21,127 [main] INFO o.a.i.c.s.RegisterManager:67 - deregister all service.
2022-12-14 02:04:21,127 [main] INFO o.a.i.db.service.DataNode:703 - IoTDB DataNode is deactivated.
2022-12-14 02:04:21,139 [main] INFO o.a.i.c.s.m.MetricService:73 - Stop metric service
2022-12-14 02:04:21,139 [main] INFO o.a.i.c.s.m.MetricService:78 - Finish stopping metric service
2022-12-14 02:04:21,140 [main] ERROR o.a.i.db.service.DataNode:676 - Stop data node error
java.lang.NullPointerException: null
at org.apache.iotdb.db.service.DataNode.stop(DataNode.java:673)
at org.apache.iotdb.db.service.DataNode.doAddNode(DataNode.java:182)
at org.apache.iotdb.db.service.DataNodeServerCommandLine.run(DataNodeServerCommandLine.java:95)
### What did you expect to see?
Nothing to preview。。
### What did you see instead?
Nothing to preview。。
### Anything else?
_No response_
### Are you willing to submit a PR?
- [ ] I'm willing to submit a PR!
Contributor guide
Research direction
Start with org.apache.iotdb.db.service.DataNode.registerInConfigNode and DataNode.stop, then inspect ConfigNodeClient around the reported line 254. Reproduce the DataNode startup failure using the logged configuration and verify that registration failure produces a clear, non-null shutdown path without the additional NullPointerException.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- databases, distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 18/100