apache / apache/datafusion-comet

Spark 4.2: ANSI arithmetic overflow tests fail under Comet

Open
#4,967 0 comments 0 reactions 0 assignees View on GitHub
area:expressions bug correctness priority:medium spark 4.2
Dominant language
Scala
Stars
1.3k
Forks
373
Avg merge
2d 4h
Merged PRs (30d)
198

Description

Several Comet arithmetic tests are skipped on Spark 4.2 (`assume(!isSpark42Plus, ...)`) pending investigation of ANSI overflow behavior differences introduced in Spark 4.2.

Affected tests in `CometExpressionSuite`:
- "unary negative integer overflow test"
- "ANSI support for add"
- "ANSI support for subtract"
- "ANSI support for multiply"

These were bulk-deferred to the generic Spark 4.2 tracker in #4126. This issue tracks the specific ANSI arithmetic overflow work so the `assume` guards can be removed.

Contributor guide

Open the contributing guide

Research direction

Start in CometExpressionSuite and inspect the four named ANSI arithmetic overflow tests, especially their assume(!isSpark42Plus, ...) guards. Run those tests on Spark 4.2 while comparing the overflow behavior introduced there; done means the guards can be removed and all four tests pass with the intended ANSI behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
scala
Domain
testing-qa
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
65/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.