integrated gluen with spark in k8s, occurred this error
- Dominant language
- Scala
- Stars
- 1.6k
- Forks
- 657
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 80
Description
### Backend
VL (Velox)
### Bug description
build jar on Debian 11 with Velox
Exception in thread "main" java.lang.UnsatisfiedLinkError: /var/data/spark-4fd11b13-e00a-4fb2-a6b4-38fe6e9a64d0/gluten-150af4fd-69eb-42a6-a392-f34123f03a42/jni/9296972f-e918-4984-87d0-7a0394fde9f4/gluten-3872844251297605475/libarrow.so.1400.1.0: libsnappy.so.1: cannot open shared object file: No such file or directory
at java.base/java.lang.ClassLoader$NativeLibrary.load0(Native Method)
at java.base/java.lang.ClassLoader$NativeLibrary.load(Unknown Source)
at java.base/java.lang.ClassLoader$NativeLibrary.loadLibrary(Unknown Source)
at java.base/java.lang.ClassLoader.loadLibrary0(Unknown Source)
at java.base/java.lang.ClassLoader.loadLibrary(Unknown Source)
at java.base/java.lang.Runtime.load0(Unknown Source)
at java.base/java.lang.System.load(Unknown Source)
at io.glutenproject.vectorized.JniLibLoader.loadFromPath0(JniLibLoader.java:89)
at io.glutenproject.vectorized.JniLibLoader.access$300(JniLibLoader.java:57)
at io.glutenproject.vectorized.JniLibLoader$JniLoadTransaction.loadWithLink(JniLibLoader.java:311)
at io.glutenproject.vectorized.JniLibLoader$JniLoadTransaction.lambda$commit$1(JniLibLoader.java:261)
at java.base/java.util.ArrayList.forEach(Unknown Source)
at io.glutenproject.vectorized.JniLibLoader$JniLoadTransaction.commit(JniLibLoader.java:257)
at io.glutenproject.backendsapi.velox.ListenerApiImpl.loadLibWithLinux(ListenerApiImpl.scala:116)
at io.glutenproject.backendsapi.velox.ListenerApiImpl.initialize(ListenerApiImpl.scala:142)
at io.glutenproject.backendsapi.velox.ListenerApiImpl.onDriverStart(ListenerApiImpl.scala:47)
at io.glutenproject.GlutenDriverPlugin.init(GlutenPlugin.scala:73)
at org.apache.spark.internal.plugin.DriverPluginContainer.$anonfun$driverPlugins$1(PluginContainer.scala:53)
at scala.collection.TraversableLike.$anonfun$flatMap$1(TraversableLike.scala:293)
at scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62)
at scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49)
at scala.collection.TraversableLike.flatMap(TraversableLike.scala:293)
at scala.collection.TraversableLike.flatMap$(TraversableLike.scala:290)
at scala.collection.AbstractTraversable.flatMap(Traversable.scala:108)
at org.apache.spark.internal.plugin.DriverPluginContainer.(PluginContainer.scala:46)
at org.apache.spark.internal.plugin.PluginContainer$.apply(PluginContainer.scala:210)
at org.apache.spark.internal.plugin.PluginContainer$.apply(PluginContainer.scala:193)
at org.apache.spark.SparkContext.(SparkContext.scala:556)
at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2690)
at org.apache.spark.sql.SparkSession$Builder.$anonfun$getOrCreate$2(SparkSession.scala:949)
at scala.Option.getOrElse(Option.scala:189)
at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:943)
at app.siteApp.SiteDataApp$.main(SiteDataApp.scala:48)
at app.siteApp.SiteDataApp.main(SiteDataApp.scala)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.base/java.lang.reflect.Method.invoke(Unknown Source)
at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:955)
at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180)
at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203)
at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90)
at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1043)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1052)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
### Spark version
None
### Spark configurations
_No response_
### System information
_No response_
### Relevant logs
```bash
Exception in thread "main" java.lang.UnsatisfiedLinkError: /var/data/spark-4fd11b13-e00a-4fb2-a6b4-38fe6e9a64d0/gluten-150af4fd-69eb-42a6-a392-f34123f03a42/jni/9296972f-e918-4984-87d0-7a0394fde9f4/gluten-3872844251297605475/libarrow.so.1400.1.0: libsnappy.so.1: cannot open shared object file: No such file or directory
at java.base/java.lang.ClassLoader$NativeLibrary.load0(Native Method)
at java.base/java.lang.ClassLoader$NativeLibrary.load(Unknown Source)
at java.base/java.lang.ClassLoader$NativeLibrary.loadLibrary(Unknown Source)
at java.base/java.lang.ClassLoader.loadLibrary0(Unknown Source)
at java.base/java.lang.ClassLoader.loadLibrary(Unknown Source)
at java.base/java.lang.Runtime.load0(Unknown Source)
at java.base/java.lang.System.load(Unknown Source)
at io.glutenproject.vectorized.JniLibLoader.loadFromPath0(JniLibLoader.java:89)
at io.glutenproject.vectorized.JniLibLoader.access$300(JniLibLoader.java:57)
at io.glutenproject.vectorized.JniLibLoader$JniLoadTransaction.loadWithLink(JniLibLoader.java:311)
at io.glutenproject.vectorized.JniLibLoader$JniLoadTransaction.lambda$commit$1(JniLibLoader.java:261)
at java.base/java.util.ArrayList.forEach(Unknown Source)
at io.glutenproject.vectorized.JniLibLoader$JniLoadTransaction.commit(JniLibLoader.java:257)
at io.glutenproject.backendsapi.velox.ListenerApiImpl.loadLibWithLinux(ListenerApiImpl.scala:116)
at io.glutenproject.backendsapi.velox.ListenerApiImpl.initialize(ListenerApiImpl.scala:142)
at io.glutenproject.backendsapi.velox.ListenerApiImpl.onDriverStart(ListenerApiImpl.scala:47)
at io.glutenproject.GlutenDriverPlugin.init(GlutenPlugin.scala:73)
at org.apache.spark.internal.plugin.DriverPluginContainer.$anonfun$driverPlugins$1(PluginContainer.scala:53)
at scala.collection.TraversableLike.$anonfun$flatMap$1(TraversableLike.scala:293)
at scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62)
at scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49)
at scala.collection.TraversableLike.flatMap(TraversableLike.scala:293)
at scala.collection.TraversableLike.flatMap$(TraversableLike.scala:290)
at scala.collection.AbstractTraversable.flatMap(Traversable.scala:108)
at org.apache.spark.internal.plugin.DriverPluginContainer.(PluginContainer.scala:46)
at org.apache.spark.internal.plugin.PluginContainer$.apply(PluginContainer.scala:210)
at org.apache.spark.internal.plugin.PluginContainer$.apply(PluginContainer.scala:193)
at org.apache.spark.SparkContext.(SparkContext.scala:556)
at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2690)
at org.apache.spark.sql.SparkSession$Builder.$anonfun$getOrCreate$2(SparkSession.scala:949)
at scala.Option.getOrElse(Option.scala:189)
at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:943)
at app.siteApp.SiteDataApp$.main(SiteDataApp.scala:48)
at app.siteApp.SiteDataApp.main(SiteDataApp.scala)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.base/java.lang.reflect.Method.invoke(Unknown Source)
at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:955)
at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180)
at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203)
at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90)
at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1043)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1052)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
```
Contributor guide
Research direction
Start at JniLibLoader.java:89 and ListenerApiImpl.scala:116, following the native-library loading path shown in the stack trace. Reproduce the Debian 11 Kubernetes build and determine why libarrow.so.1400.1.0 cannot find libsnappy.so.1; done means the application starts without this UnsatisfiedLinkError.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- debian, java, kubernetes, scala
- Domain
- backend, distributed-systems, infrastructure
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100