apache / apache/datafusion-comet

ANSI mode test coverage: re-enable stale ignored tests and add missing cases

Open
#5,076 2 comments 0 reactions 1 assignee Claimed by @rich7420 View on GitHub
area:expressions enhancement test
Dominant language
Scala
Stars
1.3k
Forks
373
Avg merge
2d 4h
Merged PRs (30d)
198

Description

### What is the problem the feature request solves?

An ANSI-mode audit against Spark 4.1.1 found several test-coverage gaps around ANSI behavior. Closing them locks in the behavior that is already correct and captures the known divergences as regression tests.

**Stale disabled tests to re-enable:**

- `spark/src/test/resources/sql-tests/expressions/array/element_at_ansi.sql` and `array/get_array_item_ansi.sql` are entirely `query ignore(#3375)` with comments claiming "Comet throws: different error message". #3375 is closed: the error-message work merged (native `GetArrayItem` re-enabled shortly after), and `CometArrayExpressionSuite` now asserts exact Spark message parity for both expressions. These files should be un-ignored.

**Missing ANSI SQL file tests** (compare the existing `math/abs_ansi.sql`, `math/arithmetic_ansi.sql`, `url/parse_url_ansi.sql` pattern):

- `round`: no ANSI case at all; also no integral negative-scale overflow case in any mode.
- `conv`: no ANSI overflow case (e.g. `conv('FFFFFFFFFFFFFFFFF', 16, 10)` throws under ANSI).
- `elt`: no out-of-range index case, no ANSI config.

**Specific untested inputs surfaced by the audit** (each is the repro for a filed bug; add as `query ignore()` until fixed, then un-ignore):

- `Long.MinValue div -1` under ANSI (IntegralDivide overflow).
- `CAST(1.0 AS DOUBLE) % CAST(0.0 AS DOUBLE)` under ANSI (float remainder by zero).
- `CAST(true AS DECIMAL(1,1))` in legacy/try mode (boolean-to-decimal overflow).
- `round(5000000000000000000L, -19)` in both modes.

**Assertion strength:** several suites assert only on message substrings, which is why wrong-exception-type divergences pass today (decimal div-by-zero, next_day, make_date, decimal-to-decimal cast). Where the error class is expected to match, assert on the exception type / `getCondition()` rather than substrings, and remove the decimal-to-decimal special case at `CometCastSuite.scala:2402-2405` once the underlying issue is fixed.

### Additional context

Found by an ANSI-mode audit of all native expressions against Spark 4.1.1.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.