apache / apache/datafusion-comet
Spark 4.2: ANSI arithmetic overflow tests fail under Comet
- Dominant language
- Scala
- Stars
- 1.3k
- Forks
- 373
- Avg merge
- 2d 4h
- Merged PRs (30d)
- 198
Description
Several Comet arithmetic tests are skipped on Spark 4.2 (`assume(!isSpark42Plus, ...)`) pending investigation of ANSI overflow behavior differences introduced in Spark 4.2.
Affected tests in `CometExpressionSuite`:
- "unary negative integer overflow test"
- "ANSI support for add"
- "ANSI support for subtract"
- "ANSI support for multiply"
These were bulk-deferred to the generic Spark 4.2 tracker in #4126. This issue tracks the specific ANSI arithmetic overflow work so the `assume` guards can be removed.
Contributor guide
Research direction
Start in CometExpressionSuite and inspect the four named ANSI arithmetic overflow tests, especially their assume(!isSpark42Plus, ...) guards. Run those tests on Spark 4.2 while comparing the overflow behavior introduced there; done means the guards can be removed and all four tests pass with the intended ANSI behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- scala
- Domain
- testing-qa
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 65/100