apache / apache/gluten

[CH] UT failures from ClickHouseAdaptiveQueryExecSuite

Open
#11,369 1 comment 0 reactions 0 assignees View on GitHub
bug triage
Dominant language
Scala
Stars
1.6k
Forks
657
Avg merge
2d 21h
Merged PRs (30d)
85

Description

### Backend

CH (ClickHouse)

### Bug description

8 UT failed within `ClickHouseAdaptiveQueryExecSuite` from the Gluten CI gluten-ut-spark35 log — they were not on our radar because they were unexpectedly skipped without #11345. All of them were introduced in this [commit][1] from the same PR #5310 "[GLUTEN-5309] Add UTs for Spark 3.5".

The failures are expected to be fixed and is currently marked as "ignore" to unblock a green Gluten CH CI.

| # | Test Name | Failure Reason |
|---|-----------|----------------|
| 1 | Gluten - Change broadcast join to merge join | List() had size 0 instead of expected size 1 |
| 2 | Gluten - Empty stage coalesced to 1-partition RDD | find(plan)(_.isInstanceOf[SortMergeJoinExecTransformer]).isDefined was false |
| 3 | Gluten - Exchange reuse | 2 did not equal 1 |
| 4 | Gluten - SPARK-30524: Do not optimize skew join if introduce additional shuffle | List() had size 0 instead of expected size 1 |
| 5 | Gluten - SPARK-33551: Do not use AQE shuffle read for repartition | List() had length 0 instead of expected length 1 |
| 6 | Gluten - SPARK-35264: Support AQE side broadcastJoin threshold | List() had size 0 instead of expected size 1 |
| 7 | Gluten - SPARK-35264: Support AQE side shuffled hash join formula | 0 did not equal 1 |
| 8 | Gluten - SPARK-37742: AQE reads invalid InMemoryRelation stats and mistakenly plans BHJ | Expected 1 BroadcastHashJoin but got multiple |

[1]: https://github.com/apache/incubator-gluten/commit/bf6f6746622cac72cab636b9083fc8383722e078

### Gluten version

main branch

### Spark version

Spark-3.5.x

### Spark configurations

_No response_

### System information

_No response_

### Relevant logs

```bash

```

Contributor guide

Open the contributing guide

Research direction

Start by running the gluten-ut-spark35 suite and reviewing ClickHouseAdaptiveQueryExecSuite, especially the eight tests introduced by commit bf6f674. Compare their listed failures with the current ignore markers. Done means all eight tests pass on Spark 3.5.x without being skipped.

Written by the indexing model from the issue text.

Assessment

Tech stack
clickhouse, scala, spark
Domain
databases, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.