apache / apache/iotdb

[Bug] FragmentInstance 20230215_122615_48442_3.1.0 is aborted by timeout

Open
#9,069 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
6.4k
Forks
1.2k
Avg merge
1d 23h
Merged PRs (30d)
115

Description

### Search before asking

- [X] I searched in the [issues](https://github.com/apache/iotdb/issues) and found nothing similar.

### Version

V1.0.0 Red Hat 4.8.5-39

### Describe the bug and provide the minimal reproduce step

distributed mode 3C3D

schema_replication_factor = 3
data_replication_factor = 2

1、user iotdb-benchmark execute pressure test
10000 device 3 sensor
write 2billion point

2、use start-cli execute this sql
SELECT last sabcdefghijkdk_1 FROM root.test.g_0.dsdfsfjsjfieir_1243;

then timeout occurs

### What did you expect to see?

it should be a simple result

### What did you see instead?

2023-02-15 20:27:15,457 [pool-27-IoTDB-ClientRPC-Processor-1627] WARN o.a.i.d.u.ErrorHandlingUtils:90 - Status code: 301, Query Statement: "SELECT last sabcdefghijkdk_1 FROM root.test.g_0.dsdfsfjsjfieir_1243". executeStatement failed
org.apache.iotdb.commons.exception.IoTDBException: org.apache.iotdb.db.mpp.execution.fragment.FragmentInstanceFailureInfo$FailureException: FragmentInstance 20230215_122615_48442_3.1.0 is aborted by timeout
at org.apache.iotdb.db.mpp.plan.execution.QueryExecution.dealWithException(QueryExecution.java:430)
at org.apache.iotdb.db.mpp.plan.execution.QueryExecution.getResult(QueryExecution.java:413)
at org.apache.iotdb.db.mpp.plan.execution.QueryExecution.getByteBufferBatchResult(QueryExecution.java:448)
at org.apache.iotdb.db.utils.QueryDataSetUtils.convertQueryResultByFetchSize(QueryDataSetUtils.java:387)
at org.apache.iotdb.db.service.thrift.impl.ClientRPCServiceImpl.lambda$static$0(ClientRPCServiceImpl.java:160)
at org.apache.iotdb.db.service.thrift.impl.ClientRPCServiceImpl.executeStatementInternal(ClientRPCServiceImpl.java:236)
at org.apache.iotdb.db.service.thrift.impl.ClientRPCServiceImpl.executeStatementV2(ClientRPCServiceImpl.java:390)
at org.apache.iotdb.service.rpc.thrift.IClientRPCService$Processor$executeStatementV2.getResult(IClientRPCService.java:3453)
at org.apache.iotdb.service.rpc.thrift.IClientRPCService$Processor$executeStatementV2.getResult(IClientRPCService.java:3433)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:38)
at org.apache.iotdb.db.service.thrift.ProcessorWithMetrics.process(ProcessorWithMetrics.java:64)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:248)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: org.apache.iotdb.db.mpp.execution.fragment.FragmentInstanceFailureInfo$FailureException: FragmentInstance 20230215_122615_48442_3.1.0 is aborted by timeout
at org.apache.iotdb.db.mpp.execution.schedule.DriverScheduler.clearDriverTask(DriverScheduler.java:246)
at org.apache.iotdb.db.mpp.execution.schedule.DriverScheduler.access$900(DriverScheduler.java:54)
at org.apache.iotdb.db.mpp.execution.schedule.DriverScheduler$Scheduler.toAborted(DriverScheduler.java:390)
at org.apache.iotdb.db.mpp.execution.schedule.DriverTaskTimeoutSentinelThread.execute(DriverTaskTimeoutSentinelThread.java:77)
at org.apache.iotdb.db.mpp.execution.schedule.AbstractDriverThread.run(AbstractDriverThread.java:74)

### Anything else?

_No response_

### Are you willing to submit a PR?

- [ ] I'm willing to submit a PR!

Contributor guide

Open the contributing guide

Research direction

Start with the reported call path in ClientRPCServiceImpl and QueryExecution, then inspect DriverScheduler and DriverTaskTimeoutSentinelThread where the timeout is raised. Run the distributed 3C3D reproduction with the stated replication factors and query, and confirm that it returns the expected simple result without aborting.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, sql
Domain
databases, distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.