[Bug] One session add repeated jar, other session DML sqls will failed
- Dominant language
- Scala
- Stars
- 2.4k
- Forks
- 1k
- PR merge metrics
- No merged PRs in 30d
Description
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
### Search before asking
- [X] I have searched in the [issues](https://github.com/apache/incubator-kyuubi/issues?q=is%3Aissue) and found no similar issues.
### Describe the bug
If the user repeatedly adds the jars in one session , the dml sql from other session will fail. this is because AddJarCommand will add jar to common sparkContext, other session task failed when down dependent jars to container .
### Affects Version(s)
master/1.7
### Kyuubi Server Log Output
_No response_
### Kyuubi Engine Log Output
```logtalk
Caused by: org.apache.spark.SparkException: File ./xxx-0.0.1.jar exists and does not match contents of hdfs://user_zzz/yyy/udf/xxx-0.0.1.jar
at org.apache.spark.util.Utils$.copyFile(Utils.scala:685)
at org.apache.spark.util.Utils$.fetchFile(Utils.scala:542)
at org.apache.spark.executor.Executor.$anonfun$updateDependencies$13(Executor.scala:963)
at org.apache.spark.executor.Executor.$anonfun$updateDependencies$13$adapted(Executor.scala:955)
```
### Kyuubi Server Configurations
_No response_
### Kyuubi Engine Configurations
_No response_
### Additional context
_No response_
### Are you willing to submit PR?
- [X] Yes. I can submit a PR independently to fix.
- [ ] Yes. I would be willing to submit a PR with guidance from the Kyuubi community to fix.
- [ ] No. I cannot submit a PR at this time.
Contributor guide
Research direction
Begin at the AddJarCommand entry point and reproduce repeated jar additions across two sessions, using the shown Spark executor dependency-fetch error as the symptom. Trace how the command reaches the shared SparkContext; done means DML from the other session no longer fails after repeated jar additions.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- scala
- Domain
- backend, distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100