apache / apache/datafusion-comet
Enable SPARK-57298 collect_set tests when Spark 4.2 SQL diff lands
- Dominant language
- Scala
- Stars
- 1.3k
- Forks
- 373
- Avg merge
- 2d 4h
- Merged PRs (30d)
- 198
Description
## Background
PR #5166 fixes #4966 and adds Comet SQL-file coverage for Spark 4.2 `collect_set` normalization.
The repository currently has no `dev/diffs/4.2*.diff` and no Spark 4.2 Spark SQL Tests job, so the corresponding upstream Spark tests cannot yet be enabled in the Comet diff.
## Task
Once a Spark 4.2 diff and CI job are added, enable these `DataFrameAggregateSuite` tests:
- `SPARK-57298: collect_set normalizes NaN and -0.0 for floating-point types`
- `SPARK-57298: collect_set normalizes NaN and -0.0 nested in complex types`
Verify both tests pass with native `collect_set`.
Related: #4966, #5166, and [review 4830920688](https://github.com/apache/datafusion-comet/pull/5166#pullrequestreview-4830920688).
Contributor guide
Assessment
This issue has not been assessed yet.