[CH] UT failures from ClickHouseAdaptiveQueryExecSuite
- Dominant language
- Scala
- Stars
- 1.6k
- Forks
- 657
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 85
Description
### Backend
CH (ClickHouse)
### Bug description
8 UT failed within `ClickHouseAdaptiveQueryExecSuite` from the Gluten CI gluten-ut-spark35 log — they were not on our radar because they were unexpectedly skipped without #11345. All of them were introduced in this [commit][1] from the same PR #5310 "[GLUTEN-5309] Add UTs for Spark 3.5".
The failures are expected to be fixed and is currently marked as "ignore" to unblock a green Gluten CH CI.
| # | Test Name | Failure Reason |
|---|-----------|----------------|
| 1 | Gluten - Change broadcast join to merge join | List() had size 0 instead of expected size 1 |
| 2 | Gluten - Empty stage coalesced to 1-partition RDD | find(plan)(_.isInstanceOf[SortMergeJoinExecTransformer]).isDefined was false |
| 3 | Gluten - Exchange reuse | 2 did not equal 1 |
| 4 | Gluten - SPARK-30524: Do not optimize skew join if introduce additional shuffle | List() had size 0 instead of expected size 1 |
| 5 | Gluten - SPARK-33551: Do not use AQE shuffle read for repartition | List() had length 0 instead of expected length 1 |
| 6 | Gluten - SPARK-35264: Support AQE side broadcastJoin threshold | List() had size 0 instead of expected size 1 |
| 7 | Gluten - SPARK-35264: Support AQE side shuffled hash join formula | 0 did not equal 1 |
| 8 | Gluten - SPARK-37742: AQE reads invalid InMemoryRelation stats and mistakenly plans BHJ | Expected 1 BroadcastHashJoin but got multiple |
[1]: https://github.com/apache/incubator-gluten/commit/bf6f6746622cac72cab636b9083fc8383722e078
### Gluten version
main branch
### Spark version
Spark-3.5.x
### Spark configurations
_No response_
### System information
_No response_
### Relevant logs
```bash
```
Contributor guide
Research direction
Start by running the gluten-ut-spark35 suite and reviewing ClickHouseAdaptiveQueryExecSuite, especially the eight tests introduced by commit bf6f674. Compare their listed failures with the current ignore markers. Done means all eight tests pass on Spark 3.5.x without being skipped.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- clickhouse, scala, spark
- Domain
- databases, testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100