[VL] Publish Gluten modified Arrow jar
- Dominant language
- Scala
- Stars
- 1.6k
- Forks
- 657
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 80
Description
The [Arrow jar](https://github.com/apache/incubator-gluten/blob/05b1e7a28fa8b2915b573496466cbe94c7baab9b/pom.xml#L71) Velox backend is using is not publicly accessible and is required to be built locally. We'd either push the modified code to upstream Arrow (cc @jinchengchenghh) or publish the modified Jar to a public accessible Maven repo.
Doing this will perhaps:
1. Avoid compiling Arrow before building Velox backend (@PHILO-HE please confirm, thanks)
2. Make it easier to verify Velox backend's Maven build without building the Arrow code in advance. Say if one doesn't ever build the Arrow jar, they can still compile Velox backend's code without "dependency not found" error.
Contributor guide
Research direction
Start with the Arrow dependency declaration in pom.xml at line 71 and inspect how the Velox backend Maven build obtains it. Determine whether the modified jar can be published to a publicly accessible Maven repository or replaced with an upstream Arrow artifact; done means the Velox backend builds without compiling Arrow locally and no dependency-not-found error occurs.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, scala
- Domain
- backend, build-system
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100