[Bug] can't connect to node TEndPoint。code:606, message:Reject write because there are too many requests need to process

### Search before asking

- [X] I searched in the [issues](https://github.com/apache/iotdb/issues) and found nothing similar.

### Version

IoTDB> show version;
+-------+-----------+
|Version| BuildInfo|
+-------+-----------+
| 1.0.0|fbbca3f-dev|
+-------+-----------+
Total line number = 1
It costs 0.002s
IoTDB> show cluster details;
+------+----------+-------+---------------+------------+-------------------+-------------+-------+-----------------+-------------------+-------+
|NodeID| NodeType| Status|InternalAddress|InternalPort|ConfigConsensusPort| RpcAddress|RpcPort|DataConsensusPort|SchemaConsensusPort|MppPort|
+------+----------+-------+---------------+------------+-------------------+-------------+-------+-----------------+-------------------+-------+
| 0|ConfigNode|Running| 127.0.0.1| 22277| 22278| | | | | |
| 1|ConfigNode|Running| 127.0.0.1| 32277| 32278| | | | | |
| 2|ConfigNode|Running| 127.0.0.1| 42277| 42278| | | | | |
| 3| DataNode|Running| 127.0.0.1| 9003| |192.168.4.165| 6667| 40010| 50010| 8777|
| 4| DataNode|Running| 127.0.0.1| 9004| |192.168.4.165| 6668| 40011| 50011| 8778|
| 5| DataNode|Running| 127.0.0.1| 9005| |192.168.4.165| 6669| 40012| 50012| 8779|
+------+----------+-------+---------------+------------+-------------------+-------------+-------+-----------------+-------------------+-------+
Total line number = 6
It costs 0.005s

### Describe the bug and provide the minimal reproduce step

1. 使用Java客户端，一直连续插入大量数据
2. 一段时间后，插入线程卡死
3. 一段时间后，插入线程恢复
4. 持续运行中，以上2，3现象反复交替出现
5. 期间，查询正常，服务器节点报错如下：

DataNode0:
2023-01-10 17:00:04,076 [pool-27-IoTDB-ClientRPC-Processor-53$20230110_085944_28488_3.1.0] WARN o.a.i.d.m.p.s.FragmentInstanceDispatcherImpl:195 - can't connect to node TEndPoint(ip:127.0.0.1, port:9005)
org.apache.thrift.TException: Error in calling method sendPlanNode, because: Error in calling method recv_sendPlanNode, because: Error in calling method receiveBase, because: java.net.SocketTimeoutException: Read timed out
at org.apache.iotdb.commons.client.sync.SyncThriftClientWithErrorHandler.intercept(SyncThriftClientWithErrorHandler.java:95)
at org.apache.iotdb.commons.client.sync.SyncDataNodeInternalServiceClient$$EnhancerByCGLIB$$2ab862aa.sendPlanNode()
at org.apache.iotdb.db.mpp.plan.scheduler.FragmentInstanceDispatcherImpl.dispatchRemote(FragmentInstanceDispatcherImpl.java:171)
at org.apache.iotdb.db.mpp.plan.scheduler.FragmentInstanceDispatcherImpl.dispatchOneInstance(FragmentInstanceDispatcherImpl.java:139)
at org.apache.iotdb.db.mpp.plan.scheduler.FragmentInstanceDispatcherImpl.dispatchWriteSync(FragmentInstanceDispatcherImpl.java:119)
at org.apache.iotdb.db.mpp.plan.scheduler.FragmentInstanceDispatcherImpl.dispatch(FragmentInstanceDispatcherImpl.java:90)
at org.apache.iotdb.db.mpp.plan.scheduler.ClusterScheduler.start(ClusterScheduler.java:106)
at org.apache.iotdb.db.mpp.plan.execution.QueryExecution.schedule(QueryExecution.java:288)
at org.apache.iotdb.db.mpp.plan.execution.QueryExecution.start(QueryExecution.java:206)
at org.apache.iotdb.db.mpp.plan.Coordinator.execute(Coordinator.java:150)
at org.apache.iotdb.db.mpp.plan.Coordinator.execute(Coordinator.java:164)
at org.apache.iotdb.db.service.thrift.impl.ClientRPCServiceImpl.insertRecords(ClientRPCServiceImpl.java:973)
at org.apache.iotdb.service.rpc.thrift.IClientRPCService$Processor$insertRecords.getResult(IClientRPCService.java:4128)
at org.apache.iotdb.service.rpc.thrift.IClientRPCService$Processor$insertRecords.getResult(IClientRPCService.java:4108)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:38)
at org.apache.iotdb.db.service.thrift.ProcessorWithMetrics.process(ProcessorWithMetrics.java:64)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:248)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:823)
Caused by: org.apache.thrift.TException: Error in calling method recv_sendPlanNode, because: Error in calling method receiveBase, because: java.net.SocketTimeoutException: Read timed out
at org.apache.iotdb.commons.client.sync.SyncThriftClientWithErrorHandler.intercept(SyncThriftClientWithErrorHandler.java:95)
at org.apache.iotdb.commons.client.sync.SyncDataNodeInternalServiceClient$$EnhancerByCGLIB$$2ab862aa.recv_sendPlanNode()
at org.apache.iotdb.mpp.rpc.thrift.IDataNodeRPCService$Client.sendPlanNode(IDataNodeRPCService.java:546)
at org.apache.iotdb.commons.client.sync.SyncDataNodeInternalServiceClient$$EnhancerByCGLIB$$2ab862aa.CGLIB$sendPlanNode$11()
at org.apache.iotdb.commons.client.sync.SyncDataNodeInternalServiceClient$$EnhancerByCGLIB$$2ab862aa$$FastClassByCGLIB$$a5fbc3b9.invoke()
at net.sf.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228)
at org.apache.iotdb.commons.client.sync.SyncThriftClientWithErrorHandler.intercept(SyncThriftClientWithErrorHandler.java:55)
... 19 common frames omitted
Caused by: org.apache.thrift.TException: Error in calling method receiveBase, because: java.net.SocketTimeoutException: Read timed out
at org.apache.iotdb.commons.client.sync.SyncThriftClientWithErrorHandler.intercept(SyncThriftClientWithErrorHandler.java:95)
at org.apache.iotdb.commons.client.sync.SyncDataNodeInternalServiceClient$$EnhancerByCGLIB$$2ab862aa.receiveBase()
at org.apache.iotdb.mpp.rpc.thrift.IDataNodeRPCService$Client.recv_sendPlanNode(IDataNodeRPCService.java:559)
at org.apache.iotdb.commons.client.sync.SyncDataNodeInternalServiceClient$$EnhancerByCGLIB$$2ab862aa.CGLIB$recv_sendPlanNode$13()
at org.apache.iotdb.commons.client.sync.SyncDataNodeInternalServiceClient$$EnhancerByCGLIB$$2ab862aa$$FastClassByCGLIB$$a5fbc3b9.invoke()
at net.sf.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228)
at org.apache.iotdb.commons.client.sync.SyncThriftClientWithErrorHandler.intercept(SyncThriftClientWithErrorHandler.java:55)
... 25 common frames omitted
Caused by: org.apache.thrift.transport.TTransportException: java.net.SocketTimeoutException: Read timed out
at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:178)
at org.apache.thrift.transport.TTransport.readAll(TTransport.java:109)
at org.apache.iotdb.rpc.TElasticFramedTransport.readFrame(TElasticFramedTransport.java:112)
at org.apache.iotdb.rpc.TElasticFramedTransport.read(TElasticFramedTransport.java:107)
at org.apache.thrift.transport.TTransport.readAll(TTransport.java:109)
at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:463)
at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:361)
at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:244)
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:77)
at org.apache.iotdb.commons.client.sync.SyncDataNodeInternalServiceClient$$EnhancerByCGLIB$$2ab862aa.CGLIB$receiveBase$174()
at org.apache.iotdb.commons.client.sync.SyncDataNodeInternalServiceClient$$EnhancerByCGLIB$$2ab862aa$$FastClassByCGLIB$$a5fbc3b9.invoke()
at net.sf.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228)
at org.apache.iotdb.commons.client.sync.SyncThriftClientWithErrorHandler.intercept(SyncThriftClientWithErrorHandler.java:55)
... 31 common frames omitted
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
at java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:176)
... 43 common frames omitted

================================================================

DataNode2:
2023-01-10 17:36:36,991 [pool-27-IoTDB-ClientRPC-Processor-42$20230110_093606_28017_5.1.0] WARN o.a.i.d.m.p.s.FragmentInstanceDispatcherImpl:236 - write locally failed. TSStatus: TSStatus(code:606, message:Reject write because there are too many requests need to process), message: Reject write because there are too many requests need to process

### What did you expect to see?

1. 数据能一直持续正常插入，而不是一会正常一会卡死

### What did you see instead?

1. 数据插入时，若此时服务器不能正常插入，直接返回异常

### Anything else?
1. 每次停顿结束后，磁盘容量占用会减少
停顿中：
df -h
/dev/sda6 3.4T 569G 2.7T 18% /nfv
停顿结束：
df -h
/dev/sda6 3.4T 517G 2.7T 16% /nfv

_No response_

### Are you willing to submit a PR?

- [X] I'm willing to submit a PR!

Commencez par reproduire le problème avec le client Java en utilisant des insertions bulk continues, puis lisez FragmentInstanceDispatcherImpl.dispatchWriteSync et le rejet d’écriture local autour du code 606. Comparez le chemin de timeout dans sendPlanNode avec le comportement de rejet signalé. Le travail est terminé lorsque les insertions continues n’alternent plus entre blocage et reprise, et qu’un serveur surchargé renvoie directement une exception.

Rédigé par le modèle d'indexation à partir du texte de l'issue.