[BUILD] `iceberg` and `iceberg-test` profiles collide on one build-helper execution id, silently dropping `src-iceberg/test` sources
- Dominant language
- Scala
- Stars
- 1.6k
- Forks
- 657
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 80
Description
### What happens
Both profiles in the root pom declare a `build-helper-maven-plugin` execution with the id `add-iceberg-test-sources`: `pom.xml:1833` under `iceberg`, `pom.xml:1884` under `iceberg-test`. Maven merges plugin executions by id, so when both profiles are active only one `` list survives.
`help:effective-pom` with `-Pbackends-clickhouse -Pspark-3.5 -Pspark-ut -Pdelta -Piceberg -Piceberg-test` resolves that execution to:
```
add-iceberg-test-sources
src-iceberg-spark35/test/scala
src-iceberg-spark35/test/java
```
The `iceberg` profile's four entries (`src-iceberg/test/{scala,java}` and `src-iceberg${iceberg.binary.version}/test/{scala,java}`) are gone. No warning is printed; the sources are simply not compiled.
### Effect on ClickHouse: the combination cannot build
`ClickHouseIcebergHiveTableSupport` lives in `backends-clickhouse/src-iceberg/test/scala`, and both Java tests under `src-iceberg-spark35/test/java` instantiate it. The class therefore leaves the source path at exactly the moment the tests join it:
```
$ ./build/mvn -Pbackends-clickhouse -Pspark-3.5 -Pscala-2.13 -Pspark-ut -Pdelta \
-Piceberg -Piceberg-test -DskipTests test-compile -pl backends-clickhouse
[ERROR] .../src-iceberg-spark35/test/java/.../TestPositionDeletesTableGluten.java:[41,18] cannot find symbol
[ERROR] symbol: class ClickHouseIcebergHiveTableSupport
```
Giving the `iceberg-test` execution a distinct id (`add-iceberg-spark-test-sources`) and changing nothing else makes the same command exit 0 and emit both `ClickHouseIcebergHiveTableSupport.class` and `TestPositionDeletesTableGluten.class`.
### Effect on Velox: harmless today, which is why nobody noticed
Nothing in `backends-velox` needs both directories at once. `src-iceberg/test` holds `VeloxIcebergSuite` and `VeloxTPCHIcebergSuite`, which run in the jobs that pass `-Piceberg` without `-Piceberg-test` (`velox_backend_x86.yml:941` and `:1013`, whose `wildcardSuites` includes `org.apache.gluten`), plus a `@EnhancedFeaturesTest` variant that `velox_backend_enhanced.yml` covers, a JUnit test surefire picks up, and one helper object. The 52 files under `src-iceberg-spark34/test` run in the three groups at `:817`, `:821`, `:825` that pass both profiles.
### Suggested fix
Rename the `iceberg-test` profile's execution so the two lists no longer overwrite each other. Both profiles are always passed together in every in-repo invocation, so the merged list is what the plugin was meant to produce.
Two things a reviewer will want to know about the blast radius. The three x86 groups and the 4.0 `test-compile` at `:1277` would newly compile `src-iceberg/test/**`, so those suites run twice across the matrix; expect longer jobs rather than new failures. And `src-iceberg${iceberg.binary.version}/test` is empty in every module, so nothing else appears on the source path.
Only `backends-velox` and `backends-clickhouse` hold files under `src-iceberg*/test`, so the change is confined to those two modules.
This blocks the ClickHouse Iceberg Java test work in #12934.
Contributor guide
Research direction
Start in the root pom.xml at the iceberg and iceberg-test build-helper-maven-plugin executions around lines 1833 and 1884. Give the two executions distinct ids, then run the reported ClickHouse test-compile command with both profiles and inspect the effective POM to confirm both source lists remain; done means the command succeeds and both named classes are emitted.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, scala
- Domain
- build-system, testing
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 86/100