[SUPPORT] Error when I add non-bundle dependency
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 111
Description
**Describe the problem you faced**
When I try to use dependency on hudi-spark3.5.x in SBT project, I get errors like
```
[error] https://repo1.maven.org/maven2/org/apache/hudi/hudi-spark3.5.x_2.12/0.15.0/hudi-spark3.5.x_${scala.binary.version}-0.15.0.jar: not found: https://repo1.maven.org/maven2/org/apache/hudi/hudi-spark3.5.x_2.12/0.15.0/hudi-spark3.5.x_${scala.binary.version}-0.15.0.jar
[error] https://repo1.maven.org/maven2/org/apache/hudi/hudi-spark-common_2.12/0.15.0/hudi-spark-common_${scala.binary.version}-0.15.0.jar: not found: https://repo1.maven.org/maven2/org/apache/hudi/hudi-spark-common_2.12/0.15.0/hudi-spark-common_${scala.binary.version}-0.15.0.jar
```
**To Reproduce**
Steps to reproduce the behavior:
1. Create a fresh minimal sbt project
2. Add build.sbt:
```
scalaVersion := "2.12.20"
name := "hudi-bundle"
version := "1.0"
lazy val root = (project in file(".")).
settings(libraryDependencies += "org.apache.hudi" %% "hudi-spark3.5.x" % "0.15.0")
```
3. Run `sbt compile`
**Expected behavior**
Project should be built
**Environment Description**
* Hudi version :
0.15.0 (1.0.2 has the same issue)
* Spark version :
3.5
* Running on Docker? (yes/no) :
No
**Stacktrace**
```
[error] lmcoursier.internal.shaded.coursier.error.FetchError$DownloadingArtifacts: Error fetching artifacts:
[error] https://repo1.maven.org/maven2/org/apache/hudi/hudi-spark3.5.x_2.12/0.15.0/hudi-spark3.5.x_${scala.binary.version}-0.15.0.jar: not found: https://repo1.maven.org/maven2/org/apache/hudi/hudi-spark3.5.x_2.12/0.15.0/hudi-spark3.5.x_${scala.binary.version}-0.15.0.jar
[error] https://repo1.maven.org/maven2/org/apache/hudi/hudi-spark-common_2.12/0.15.0/hudi-spark-common_${scala.binary.version}-0.15.0.jar: not found: https://repo1.maven.org/maven2/org/apache/hudi/hudi-spark-common_2.12/0.15.0/hudi-spark-common_${scala.binary.version}-0.15.0.jar
[error]
[error] at lmcoursier.internal.shaded.coursier.Artifacts$.$anonfun$fetchArtifacts$9(Artifacts.scala:365)
[error] at lmcoursier.internal.shaded.coursier.util.Task$.$anonfun$flatMap$extension$1(Task.scala:14)
[error] at lmcoursier.internal.shaded.coursier.util.Task$.$anonfun$flatMap$extension$1$adapted(Task.scala:14)
[error] at lmcoursier.internal.shaded.coursier.util.Task$.wrap(Task.scala:82)
[error] at lmcoursier.internal.shaded.coursier.util.Task$.$anonfun$flatMap$2(Task.scala:14)
[error] at scala.concurrent.Future.$anonfun$flatMap$1(Future.scala:307)
[error] at scala.concurrent.impl.Promise.$anonfun$transformWith$1(Promise.scala:51)
[error] at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:74)
[error] at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
[error] at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
[error] at java.base/java.lang.Thread.run(Thread.java:1583)
[error] Caused by: lmcoursier.internal.shaded.coursier.cache.ArtifactError$NotFound: not found: https://repo1.maven.org/maven2/org/apache/hudi/hudi-spark3.5.x_2.12/0.15.0/hudi-spark3.5.x_${scala.binary.version}-0.15.0.jar
[error] at lmcoursier.internal.shaded.coursier.cache.internal.Downloader$Blocking$.doDownload(Downloader.scala:239)
[error] at lmcoursier.internal.shaded.coursier.cache.internal.Downloader$Blocking$.$anonfun$remote$2(Downloader.scala:423)
[error] at lmcoursier.internal.shaded.coursier.cache.CacheLocks$.loop$1(CacheLocks.scala:94)
[error] at lmcoursier.internal.shaded.coursier.cache.CacheLocks$.withLockOr(CacheLocks.scala:117)
[error] at lmcoursier.internal.shaded.coursier.cache.internal.Downloader$Blocking$.$anonfun$remote$1(Downloader.scala:426)
[error] at lmcoursier.internal.shaded.coursier.cache.internal.Downloader$.$anonfun$downloading$2(Downloader.scala:776)
[error] at lmcoursier.internal.shaded.coursier.cache.CacheLocks$.withUrlLock(CacheLocks.scala:136)
[error] at lmcoursier.internal.shaded.coursier.cache.internal.Downloader$.$anonfun$downloading$1(Downloader.scala:776)
[error] at lmcoursier.internal.shaded.coursier.cache.internal.Retry.loop$1(Retry.scala:23)
[error] at lmcoursier.internal.shaded.coursier.cache.internal.Retry.retryOpt(Retry.scala:39)
[error] at lmcoursier.internal.shaded.coursier.cache.internal.Downloader$.coursier$cache$internal$Downloader$$downloading(Downloader.scala:812)
[error] at lmcoursier.internal.shaded.coursier.cache.internal.Downloader$Blocking$.remote(Downloader.scala:428)
[error] at lmcoursier.internal.shaded.coursier.cache.internal.Downloader.$anonfun$remote$6(Downloader.scala:546)
[error] at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
[error] at scala.util.Success.$anonfun$map$1(Try.scala:255)
[error] at scala.util.Success.map(Try.scala:213)
[error] at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
[error] at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:42)
[error] at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:74)
[error] at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
[error] at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
[error] at java.base/java.lang.Thread.run(Thread.java:1583)
```
When I change dependency to `"org.apache.hudi" % "hudi-spark3.5.x_2.12" % "0.15.0"`, I get the same result with the same errors.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the minimal build.sbt and reproduce the failure using sbt compile. Compare the published Maven artifacts and metadata for hudi-spark3.5.x and hudi-spark-common with the literal ${scala.binary.version} paths in the error. Done means the dependency resolves from Maven Central and the sample project builds successfully.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- scala
- Domain
- build-system, data-engineering
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100