apache / apache/iotdb

Thrift Error occurred during processing of message. org.apache.thrift.transport.TTransportException: java.net.SocketException: Connection reset [Bug]

Open
#8,906 5 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
6.4k
Forks
1.2k
Avg merge
1d 23h
Merged PRs (30d)
115

Description

### Search before asking

- [X] I searched in the [issues](https://github.com/apache/iotdb/issues) and found nothing similar.

### Version

I am running IOTDB 0.13.3 version on a centos 7 machine.

### Describe the bug and provide the minimal reproduce step

I have inserted approx 7000 timeseries` into the databases and trying to get all of them based on a filter time condition. I have increased the `max_deduplicated_path_num` value to 10000. When I am trying to get the `timeseries` I am getting the below error in `logs/log_all.log`

`2023-01-25 15:08:21,520 [pool-19-IoTDB-RPC-Client-6] ERROR o.a.t.s.TThreadPoolServer$WorkerProcess:258 - Thrift Error occurred during processing of message.
org.apache.thrift.transport.TTransportException: java.net.SocketException: Connection reset
at org.apache.thrift.transport.TIOStreamTransport.write(TIOStreamTransport.java:196)
at org.apache.iotdb.rpc.TElasticFramedTransport.flush(TElasticFramedTransport.java:135)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:60)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:38)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:248)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: java.net.SocketException: Connection reset
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:115)
at java.net.SocketOutputStream.write(SocketOutputStream.java:155)
at java.io.BufferedOutputStream.write(BufferedOutputStream.java:122)
at org.apache.thrift.transport.TIOStreamTransport.write(TIOStreamTransport.java:194)
... 7 common frames omitted
2023-01-25 15:08:21,520 [pool-19-IoTDB-RPC-Client-6] INFO o.a.i.d.s.b.ServiceProvider:232 - Session-6 is closing`

I am getting the below error in the client:
`15:07:53.708 [WorkerThread0] DEBUG org.apache.iotdb.rpc.AutoResizingBuffer - org.apache.iotdb.rpc.AutoResizingBuffer@9718f40 expand from 65536 t$
98304, request: 65539
15:07:53.712 [WorkerThread0] DEBUG org.apache.iotdb.rpc.AutoResizingBuffer - org.apache.iotdb.rpc.AutoResizingBuffer@9718f40 expand from 98304 t$
147456, request: 98306
15:07:53.713 [WorkerThread0] DEBUG org.apache.iotdb.rpc.AutoResizingBuffer - org.apache.iotdb.rpc.AutoResizingBuffer@9718f40 expand from 147456 $
o 221184, request: 147478
15:07:53.715 [WorkerThread0] DEBUG org.apache.iotdb.rpc.AutoResizingBuffer - org.apache.iotdb.rpc.AutoResizingBuffer@9718f40 expand from 221184 $
o 331776, request: 221188
Wed Jan 25 15:08:24 IST 2023 - Fail to reconnect to server. Please check server status`

I am using Java Native API to query the data.
Is there anything I am missing? Can anyone help me in fixing this?

### What did you expect to see?

I expect the program to run without any error and gives me the result of the query

### What did you see instead?

I got exception in the server side.

### Anything else?

_No response_

### Are you willing to submit a PR?

- [ ] I'm willing to submit a PR!

Contributor guide

Open the contributing guide

Research direction

Start with logs/log_all.log and the client output showing AutoResizingBuffer expansion, then trace the server-side TElasticFramedTransport failure and the Java Native API query. Reproduce the query against approximately 7,000 timeseries on IoTDB 0.13.3 and determine the cause of the connection reset; done means the failure is explained and the query completes without the reported errors.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.