apache / apache/gluten

[BUILD] `iceberg` and `iceberg-test` profiles collide on one build-helper execution id, silently dropping `src-iceberg/test` sources

Open Beginner friendly
#12,933 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Scala
Stars
1.6k
Forks
657
Avg merge
2d 14h
Merged PRs (30d)
80

Description

### What happens

Both profiles in the root pom declare a `build-helper-maven-plugin` execution with the id `add-iceberg-test-sources`: `pom.xml:1833` under `iceberg`, `pom.xml:1884` under `iceberg-test`. Maven merges plugin executions by id, so when both profiles are active only one `` list survives.

`help:effective-pom` with `-Pbackends-clickhouse -Pspark-3.5 -Pspark-ut -Pdelta -Piceberg -Piceberg-test` resolves that execution to:

```
add-iceberg-test-sources
src-iceberg-spark35/test/scala
src-iceberg-spark35/test/java
```

The `iceberg` profile's four entries (`src-iceberg/test/{scala,java}` and `src-iceberg${iceberg.binary.version}/test/{scala,java}`) are gone. No warning is printed; the sources are simply not compiled.

### Effect on ClickHouse: the combination cannot build

`ClickHouseIcebergHiveTableSupport` lives in `backends-clickhouse/src-iceberg/test/scala`, and both Java tests under `src-iceberg-spark35/test/java` instantiate it. The class therefore leaves the source path at exactly the moment the tests join it:

```
$ ./build/mvn -Pbackends-clickhouse -Pspark-3.5 -Pscala-2.13 -Pspark-ut -Pdelta \
-Piceberg -Piceberg-test -DskipTests test-compile -pl backends-clickhouse
[ERROR] .../src-iceberg-spark35/test/java/.../TestPositionDeletesTableGluten.java:[41,18] cannot find symbol
[ERROR] symbol: class ClickHouseIcebergHiveTableSupport
```

Giving the `iceberg-test` execution a distinct id (`add-iceberg-spark-test-sources`) and changing nothing else makes the same command exit 0 and emit both `ClickHouseIcebergHiveTableSupport.class` and `TestPositionDeletesTableGluten.class`.

### Effect on Velox: harmless today, which is why nobody noticed

Nothing in `backends-velox` needs both directories at once. `src-iceberg/test` holds `VeloxIcebergSuite` and `VeloxTPCHIcebergSuite`, which run in the jobs that pass `-Piceberg` without `-Piceberg-test` (`velox_backend_x86.yml:941` and `:1013`, whose `wildcardSuites` includes `org.apache.gluten`), plus a `@EnhancedFeaturesTest` variant that `velox_backend_enhanced.yml` covers, a JUnit test surefire picks up, and one helper object. The 52 files under `src-iceberg-spark34/test` run in the three groups at `:817`, `:821`, `:825` that pass both profiles.

### Suggested fix

Rename the `iceberg-test` profile's execution so the two lists no longer overwrite each other. Both profiles are always passed together in every in-repo invocation, so the merged list is what the plugin was meant to produce.

Two things a reviewer will want to know about the blast radius. The three x86 groups and the 4.0 `test-compile` at `:1277` would newly compile `src-iceberg/test/**`, so those suites run twice across the matrix; expect longer jobs rather than new failures. And `src-iceberg${iceberg.binary.version}/test` is empty in every module, so nothing else appears on the source path.

Only `backends-velox` and `backends-clickhouse` hold files under `src-iceberg*/test`, so the change is confined to those two modules.

This blocks the ClickHouse Iceberg Java test work in #12934.

Contributor guide

Open the contributing guide

Research direction

Start in the root pom.xml at the iceberg and iceberg-test build-helper-maven-plugin executions around lines 1833 and 1884. Give the two executions distinct ids, then run the reported ClickHouse test-compile command with both profiles and inspect the effective POM to confirm both source lists remain; done means the command succeeds and both named classes are emitted.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, scala
Domain
build-system, testing
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
86/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.