apache / apache/datafusion-comet

Native divide-by-zero in a dispatched ScalaUDF surfaces CometNativeException instead of SparkArithmeticException

Open
#4,517 1 comment 0 reactions 1 assignee Claimed by @0lai0 View on GitHub
area:expressions priority:medium
Dominant language
Scala
Stars
1.3k
Forks
373
Avg merge
2d 4h
Merged PRs (30d)
198

Description

When a `ScalaUDF` dispatched into the native plan (#4514) participates in an expression that raises a divide-by-zero under ANSI mode (e.g. `1/udf(a)`), the error surfaces as `org.apache.comet.CometNativeException` ("DivideByZero") instead of the Spark `org.apache.spark.SparkArithmeticException` with error class `DIVIDE_BY_ZERO`. DataFusion wraps the typed error so the JNI bridge cannot downcast it back to the matching Spark exception.

This is observable in Spark's `SQLQueryTestSuite` golden test `udf/postgreSQL/udf-select_having.sql` (query `SELECT 1 AS one FROM test_having WHERE 1/udf(a) = 1 HAVING 1 < 2`). Same error category, different surface.

Follow-on from #4514. We should preserve the typed Spark exception (exception class plus error class) end to end so native evaluation matches Spark's ANSI error behavior.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.