apache / apache/hudi

Hoist maven-shade-plugin dependency-reduced POM flags to the root pom

Open
#19,466 0 comments 0 reactions 1 assignee Claimed by @rangareddy View on GitHub
area:build priority:medium type:devtask
Dominant language
Java
Stars
6.2k
Forks
2.5k
Avg merge
2d 8h
Merged PRs (30d)
111

Description

## Task Description

**What needs to be done:**

Follow-up from #19433. Move the three `maven-shade-plugin` flags that control the published POM -- `createDependencyReducedPom`, `promoteTransitiveDependencies` and `dependencyReducedPomLocation` -- into the shared shade `` that already exists in the root `pom.xml`, delete the per-module copies, and keep a single documented override in `hudi-io`.

The mechanism is already there and unused for this purpose: the root pom declares `maven-shade-plugin` in `` with a shared `` carrying `artifactSet` and `relocations`, which every bundle inherits and extends via `combine.children="append"`. These three flags simply were never put in it.

**Why this task is needed:**

#19433 set `createDependencyReducedPom=true` and `promoteTransitiveDependencies=true` on seven bundles. That was the right scope for that PR, but it leaves the repo with no single convention. State once it merges:

| modules | `createDependencyReducedPom` | `promoteTransitiveDependencies` |
| --- | --- | --- |
| the 7 from #19433 | `true` (explicit) | `true` (explicit) |
| `hudi-timeline-server-bundle` | `true` (explicit) | unset, so `false` |
| the other 8 `packaging/` bundles | unset, so `true` | unset, so `false` |
| `hudi-examples-k8s` | unset, so `true` | unset, so `false` |
| `hudi-io` | `false` (explicit) | unset, so `false` |

18 modules run shade -- the 16 under `packaging/` plus `hudi-io` and `hudi-examples-k8s` -- and no two of them agree on the full set of flags. Adding promotion to nine more POMs individually would make that a tenth copy of the same drift rather than fixing it.

Four specifics worth capturing before anyone picks this up:

**1. `hudi-io` must keep `createDependencyReducedPom=false`.** It is the only module in the repo using `shadedArtifactAttached=true`: the shaded jar ships under a `shaded` classifier and the main artifact stays unshaded, so its POM has to keep declaring `protobuf-java`. Reducing it would strip a dependency the primary jar genuinely needs. The invariant is **`false` is correct iff the shaded jar is attached under a classifier**, not "false is always wrong". This belongs as a comment in `hudi-io/pom.xml` so it does not get "fixed" by someone reading #19433 later.

**2. For the other nine the gap is promotion, not reduction.** They already publish reduced POMs, since shade defaults to `true`. Without promotion shade drops the absorbed dependency *and* everything that reached the project only through it, including artifacts that are not in the jar -- so the published POM loses real runtime dependencies.

**3. Five of the nine carry the real exposure.** The "already lands on a classpath that supplies everything" argument covers `hudi-spark-bundle`, `hudi-utilities-bundle`, `hudi-utilities-slim-bundle` and `hudi-flink-bundle`. It does not cover `hudi-cli-bundle`, `hudi-kafka-connect-bundle`, `hudi-integ-test-bundle`, `hudi-metaserver-server-bundle` or `hudi-timeline-server-bundle`. Verify those five first.

**4. `hudi-timeline-server-bundle` is the only bundle not setting `dependencyReducedPomLocation`**, so it writes `dependency-reduced-pom.xml` to the module basedir instead of `target/`. Gitignored and the published POM is identical, so purely cosmetic, but it goes away for free with the hoist.

`hudi-examples-k8s` inherits from the root as well and needs its own decision, since it is examples rather than a published bundle.

**Also in scope, since the sweep touches every packaging POM anyway:** drop the dead relocation entry in `packaging/hudi-aws-bundle/pom.xml`.

```xml

org.apache.httpcomponents.
org.apache.hudi.aws.org.apache.httpcomponents.

```

`org.apache.httpcomponents` is the groupId, not a Java package -- those classes live under `org.apache.http.*`, so this rule matches nothing. They are relocated, but by the inherited root rule `org.apache.http.` -> `org.apache.hudi.org.apache.http.`, which applies because `` merges the parent config. Confirmed in the built jar: everything under `org/apache/hudi/org/apache/http/`, nothing unrelocated, nothing under `org/apache/hudi/aws/`. Deleting it is a no-op on jar contents.

**Suggested approach:**

1. Add the three flags to the root shade ``.
2. Delete the per-module copies from the 16 `packaging/*` POMs.
3. Keep `createDependencyReducedPom=false` in `hudi-io`, with a comment giving the classifier reason.
4. Decide explicitly what `hudi-examples-k8s` should do.
5. Delete the dead `org.apache.httpcomponents.` relocation in `hudi-aws-bundle`.

Doing it this way also means the next bundle added under `packaging/` inherits the correct behaviour instead of relying on someone remembering to set three flags.

**Verification:**

Per bundle, using the install-then-resolve recipe from #19433. An in-reactor build never sees the reduced POM, because shade mutates `project.getOriginalModel()` and calls `setFile(reducedPom)` while Maven's `ReactorReader` serves the effective model, which shade does not touch. A stale un-reduced POM in `~/.m2` also masks it.

```
mvn install -pl packaging/ -DskipTests # publishes the reduced POM
mvn package -pl -DskipTests # no -am, resolves from the repository
```

Then compare `target/dependency-reduced-pom.xml` against the bundle's effective artifactSet -- its own includes plus the inherited root includes -- and confirm jar listings are unchanged against the base ref. That per-bundle verification is the real cost here, and is why #19433 deliberately stopped at seven.

## Task Type

Code improvement/refactoring

## Related Issues

**Parent feature issue:** #16407
**Related issues:** #19433

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.