apache / apache/gluten

[VL] Publish Gluten modified Arrow jar

Open
#8,228 11 comments 3 reactions 0 assignees View on GitHub
triage
Dominant language
Scala
Stars
1.6k
Forks
657
Avg merge
2d 14h
Merged PRs (30d)
80

Description

The [Arrow jar](https://github.com/apache/incubator-gluten/blob/05b1e7a28fa8b2915b573496466cbe94c7baab9b/pom.xml#L71) Velox backend is using is not publicly accessible and is required to be built locally. We'd either push the modified code to upstream Arrow (cc @jinchengchenghh) or publish the modified Jar to a public accessible Maven repo.

Doing this will perhaps:

1. Avoid compiling Arrow before building Velox backend (@PHILO-HE please confirm, thanks)
2. Make it easier to verify Velox backend's Maven build without building the Arrow code in advance. Say if one doesn't ever build the Arrow jar, they can still compile Velox backend's code without "dependency not found" error.

Contributor guide

Open the contributing guide

Research direction

Start with the Arrow dependency declaration in pom.xml at line 71 and inspect how the Velox backend Maven build obtains it. Determine whether the modified jar can be published to a publicly accessible Maven repository or replaced with an upstream Arrow artifact; done means the Velox backend builds without compiling Arrow locally and no dependency-not-found error occurs.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, scala
Domain
backend, build-system
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.