[BUILD] Allow auron-build.sh to build with an existing Maven and forward arbitrary Maven options
- Dominant language
- Rust
- Stars
- 1.8k
- Forks
- 241
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 22
Description
**Is your feature request related to a problem? Please describe.**
Calling `auron-build.sh` from another build script is awkward, and on a machine without internet access it doesn't work at all. Three things get in the way.
It always builds through `build/mvn`, which downloads Maven from the Apache mirrors. That download fails in an offline or airgapped build, and there's no way to point the script at a Maven that's already installed. `build/mvn` does honour `MVN_HOME` and `SKIP_MVN_VERSION_CHECK`, but nothing at the top level lets a caller skip the wrapper.
Most Maven options still can't be passed through. #2227 fixed `-D` handling with `MVN_D_ARGS`, but `-P`, `-B` and long options like `--no-transfer-progress` are either swallowed by the `break` in the `-*` handler or rejected outright:
```
$ sh auron-build.sh --release --sparkver 3.5 --scalaver 2.12 --no-transfer-progress
[ERROR] Unknown option: --no-transfer-progress
```
The goal is hardcoded to `install`, so building a distribution also writes every module into `~/.m2`. Nothing in the build needs that, since the assembly module is in the root reactor and its artifacts are read out of `target/`. It causes real trouble for anyone building several variants at one version: the intermediate modules carry no Spark version in their artifactId (`auron-common_2.12`, `spark-extension_2.12`, `auron-core`), so building Spark 3.1 and 3.5 at the same `project.version` writes two different jars over a single GAV. Whichever ran last is what every later build on that machine resolves.
**Describe the solution you'd like**
- `--mvn ` to build with a given Maven instead of `build/mvn`. It should take either a path or a command on `PATH`, and say so clearly when it can't find either.
- `--` as an end-of-options marker, with everything after it passed to Maven untouched. This sits alongside the existing `-D` handling rather than replacing it.
- `--goal ` for the Maven goal, still defaulting to `install` so nothing changes for current callers, but letting a caller run `package` and leave the local repository alone.
That's enough for a wrapper script to drive the whole build:
```
auron-build.sh --release --mvn "$MVN" --goal package --sparkver 3.5 --scalaver 2.12 \
-- -Pceleborn-0.6 -B --no-transfer-progress
```
**Describe alternatives you've considered**
Setting `MVN_HOME` and letting `build/mvn` pick it up. This works for the compile step, but `build/mvn` is still what gets invoked, so the version check and mirror logic stay in the path. A flag is clearer about intent and easier to pass down from a parent script.
Teaching `MVN_D_ARGS` to collect `-P` and long options too. Fewer flags, but then the collector has to know every Maven option worth forwarding, and that list keeps growing. `--` is the usual answer and needs no upkeep as Maven changes.
Leaving `--goal` alone and always running `install`. That's the status quo, and it's what makes the multi-variant case above quietly wrong. Pointing `-Dmaven.repo.local` at a scratch directory works around it, but throws away the cache of every third-party dependency at the same time.
Changing the default to `package`. Tidier, but it would break any caller relying on the install side effect, so the default is better left as it is.
The `--` part is useful on its own and could land separately as a follow-up to #2227 if you'd rather keep the changes small.
**Additional context**
The `--` separator would also make `auron-build.sh` usable as a build step inside a larger script, which is the case that surfaced all three of these.
Contributor guide
Research direction
Start with auron-build.sh and the existing build/mvn wrapper; trace how options are parsed and how the Maven goal is selected. Exercise the command shown in the issue, including --mvn, --goal package, and arguments after --. Done means an existing Maven can be selected, arbitrary Maven options reach Maven unchanged, package avoids install, and existing callers still default to install.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- shell
- Domain
- build-system, cli, tooling
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 68/100