apache / apache/gluten

integrated gluen with spark in k8s, occurred this error

Open
#4,186 0 comments 0 reactions 0 assignees View on GitHub
bug triage
Dominant language
Scala
Stars
1.6k
Forks
657
Avg merge
2d 14h
Merged PRs (30d)
80

Description

### Backend

VL (Velox)

### Bug description

build jar on Debian 11 with Velox

Exception in thread "main" java.lang.UnsatisfiedLinkError: /var/data/spark-4fd11b13-e00a-4fb2-a6b4-38fe6e9a64d0/gluten-150af4fd-69eb-42a6-a392-f34123f03a42/jni/9296972f-e918-4984-87d0-7a0394fde9f4/gluten-3872844251297605475/libarrow.so.1400.1.0: libsnappy.so.1: cannot open shared object file: No such file or directory
at java.base/java.lang.ClassLoader$NativeLibrary.load0(Native Method)
at java.base/java.lang.ClassLoader$NativeLibrary.load(Unknown Source)
at java.base/java.lang.ClassLoader$NativeLibrary.loadLibrary(Unknown Source)
at java.base/java.lang.ClassLoader.loadLibrary0(Unknown Source)
at java.base/java.lang.ClassLoader.loadLibrary(Unknown Source)
at java.base/java.lang.Runtime.load0(Unknown Source)
at java.base/java.lang.System.load(Unknown Source)
at io.glutenproject.vectorized.JniLibLoader.loadFromPath0(JniLibLoader.java:89)
at io.glutenproject.vectorized.JniLibLoader.access$300(JniLibLoader.java:57)
at io.glutenproject.vectorized.JniLibLoader$JniLoadTransaction.loadWithLink(JniLibLoader.java:311)
at io.glutenproject.vectorized.JniLibLoader$JniLoadTransaction.lambda$commit$1(JniLibLoader.java:261)
at java.base/java.util.ArrayList.forEach(Unknown Source)
at io.glutenproject.vectorized.JniLibLoader$JniLoadTransaction.commit(JniLibLoader.java:257)
at io.glutenproject.backendsapi.velox.ListenerApiImpl.loadLibWithLinux(ListenerApiImpl.scala:116)
at io.glutenproject.backendsapi.velox.ListenerApiImpl.initialize(ListenerApiImpl.scala:142)
at io.glutenproject.backendsapi.velox.ListenerApiImpl.onDriverStart(ListenerApiImpl.scala:47)
at io.glutenproject.GlutenDriverPlugin.init(GlutenPlugin.scala:73)
at org.apache.spark.internal.plugin.DriverPluginContainer.$anonfun$driverPlugins$1(PluginContainer.scala:53)
at scala.collection.TraversableLike.$anonfun$flatMap$1(TraversableLike.scala:293)
at scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62)
at scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49)
at scala.collection.TraversableLike.flatMap(TraversableLike.scala:293)
at scala.collection.TraversableLike.flatMap$(TraversableLike.scala:290)
at scala.collection.AbstractTraversable.flatMap(Traversable.scala:108)
at org.apache.spark.internal.plugin.DriverPluginContainer.(PluginContainer.scala:46)
at org.apache.spark.internal.plugin.PluginContainer$.apply(PluginContainer.scala:210)
at org.apache.spark.internal.plugin.PluginContainer$.apply(PluginContainer.scala:193)
at org.apache.spark.SparkContext.(SparkContext.scala:556)
at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2690)
at org.apache.spark.sql.SparkSession$Builder.$anonfun$getOrCreate$2(SparkSession.scala:949)
at scala.Option.getOrElse(Option.scala:189)
at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:943)
at app.siteApp.SiteDataApp$.main(SiteDataApp.scala:48)
at app.siteApp.SiteDataApp.main(SiteDataApp.scala)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.base/java.lang.reflect.Method.invoke(Unknown Source)
at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:955)
at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180)
at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203)
at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90)
at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1043)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1052)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)

### Spark version

None

### Spark configurations

_No response_

### System information

_No response_

### Relevant logs

```bash
Exception in thread "main" java.lang.UnsatisfiedLinkError: /var/data/spark-4fd11b13-e00a-4fb2-a6b4-38fe6e9a64d0/gluten-150af4fd-69eb-42a6-a392-f34123f03a42/jni/9296972f-e918-4984-87d0-7a0394fde9f4/gluten-3872844251297605475/libarrow.so.1400.1.0: libsnappy.so.1: cannot open shared object file: No such file or directory
at java.base/java.lang.ClassLoader$NativeLibrary.load0(Native Method)
at java.base/java.lang.ClassLoader$NativeLibrary.load(Unknown Source)
at java.base/java.lang.ClassLoader$NativeLibrary.loadLibrary(Unknown Source)
at java.base/java.lang.ClassLoader.loadLibrary0(Unknown Source)
at java.base/java.lang.ClassLoader.loadLibrary(Unknown Source)
at java.base/java.lang.Runtime.load0(Unknown Source)
at java.base/java.lang.System.load(Unknown Source)
at io.glutenproject.vectorized.JniLibLoader.loadFromPath0(JniLibLoader.java:89)
at io.glutenproject.vectorized.JniLibLoader.access$300(JniLibLoader.java:57)
at io.glutenproject.vectorized.JniLibLoader$JniLoadTransaction.loadWithLink(JniLibLoader.java:311)
at io.glutenproject.vectorized.JniLibLoader$JniLoadTransaction.lambda$commit$1(JniLibLoader.java:261)
at java.base/java.util.ArrayList.forEach(Unknown Source)
at io.glutenproject.vectorized.JniLibLoader$JniLoadTransaction.commit(JniLibLoader.java:257)
at io.glutenproject.backendsapi.velox.ListenerApiImpl.loadLibWithLinux(ListenerApiImpl.scala:116)
at io.glutenproject.backendsapi.velox.ListenerApiImpl.initialize(ListenerApiImpl.scala:142)
at io.glutenproject.backendsapi.velox.ListenerApiImpl.onDriverStart(ListenerApiImpl.scala:47)
at io.glutenproject.GlutenDriverPlugin.init(GlutenPlugin.scala:73)
at org.apache.spark.internal.plugin.DriverPluginContainer.$anonfun$driverPlugins$1(PluginContainer.scala:53)
at scala.collection.TraversableLike.$anonfun$flatMap$1(TraversableLike.scala:293)
at scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62)
at scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49)
at scala.collection.TraversableLike.flatMap(TraversableLike.scala:293)
at scala.collection.TraversableLike.flatMap$(TraversableLike.scala:290)
at scala.collection.AbstractTraversable.flatMap(Traversable.scala:108)
at org.apache.spark.internal.plugin.DriverPluginContainer.(PluginContainer.scala:46)
at org.apache.spark.internal.plugin.PluginContainer$.apply(PluginContainer.scala:210)
at org.apache.spark.internal.plugin.PluginContainer$.apply(PluginContainer.scala:193)
at org.apache.spark.SparkContext.(SparkContext.scala:556)
at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2690)
at org.apache.spark.sql.SparkSession$Builder.$anonfun$getOrCreate$2(SparkSession.scala:949)
at scala.Option.getOrElse(Option.scala:189)
at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:943)
at app.siteApp.SiteDataApp$.main(SiteDataApp.scala:48)
at app.siteApp.SiteDataApp.main(SiteDataApp.scala)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.base/java.lang.reflect.Method.invoke(Unknown Source)
at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:955)
at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180)
at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203)
at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90)
at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1043)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1052)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
```

Contributor guide

Open the contributing guide

Research direction

Start at JniLibLoader.java:89 and ListenerApiImpl.scala:116, following the native-library loading path shown in the stack trace. Reproduce the Debian 11 Kubernetes build and determine why libarrow.so.1400.1.0 cannot find libsnappy.so.1; done means the application starts without this UnsatisfiedLinkError.

Written by the indexing model from the issue text.

Assessment

Tech stack
debian, java, kubernetes, scala
Domain
backend, distributed-systems, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.