apache / apache/iotdb

Previous exception in task: org.apache.thrift.transport.TSocket.<init>(Lorg/apache/thrift/TConfiguration;Ljava/lang/String;II)V

Open
#13,187 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
6.4k
Forks
1.2k
Avg merge
1d 23h
Merged PRs (30d)
115

Description

spark2.4 访问 IoTDB-1.31.1 发生异常,thrift-0.14.2,详细错误入下,
org.apache.spark.util.TaskCompletionListenerException: null
Previous exception in task: org.apache.thrift.transport.TSocket.(Lorg/apache/thrift/TConfiguration;Ljava/lang/String;II)V
org.apache.iotdb.rpc.RpcTransportFactory.getTransport(RpcTransportFactory.java:82)
org.apache.iotdb.jdbc.IoTDBConnection.openTransport(IoTDBConnection.java:481)
org.apache.iotdb.jdbc.IoTDBConnection.(IoTDBConnection.java:113)
org.apache.iotdb.jdbc.IoTDBDriver.connect(IoTDBDriver.java:64)
java.sql.DriverManager.getConnection(DriverManager.java:664)
java.sql.DriverManager.getConnection(DriverManager.java:247)
org.apache.iotdb.spark.db.IoTDBRDD$$anon$1.(IoTDBRDD.scala:74)
org.apache.iotdb.spark.db.IoTDBRDD.compute(IoTDBRDD.scala:54)
org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
org.apache.spark.rdd.RDD.iterator(RDD.scala:288)
org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
org.apache.spark.rdd.RDD.iterator(RDD.scala:288)
org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
org.apache.spark.rdd.RDD.iterator(RDD.scala:288)
org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
org.apache.spark.rdd.RDD.iterator(RDD.scala:288)
org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:99)
org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:55)
org.apache.spark.scheduler.Task.run(Task.scala:121)
org.apache.spark.executor.Executor$TaskRunner$$anonfun$11.apply(Executor.scala:407)
org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1408)
org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:413)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
java.lang.Thread.run(Thread.java:748)
at org.apache.spark.TaskContextImpl.invokeListeners(TaskContextImpl.scala:138)
at org.apache.spark.TaskContextImpl.markTaskCompleted(TaskContextImpl.scala:116)
at org.apache.spark.scheduler.Task.run(Task.scala:131)
at org.apache.spark.executor.Executor$TaskRunner$$anonfun$11.apply(Executor.scala:407)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1408)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:413)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

Contributor guide

Open the contributing guide

Research direction

Start with org.apache.iotdb.rpc.RpcTransportFactory.getTransport at RpcTransportFactory.java:82 and the connection path through IoTDBConnection.java:481 and IoTDBRDD.scala:74. Compare the Spark 2.4, IoTDB 1.31.1, and thrift-0.14.2 dependency expectations, then reproduce the reported job failure; done means the Spark access no longer raises the TSocket constructor error.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, spark
Domain
databases, distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.