apache / apache/datafusion-comet
Native divide-by-zero in a dispatched ScalaUDF surfaces CometNativeException instead of SparkArithmeticException
- Dominant language
- Scala
- Stars
- 1.3k
- Forks
- 373
- Avg merge
- 2d 4h
- Merged PRs (30d)
- 198
Description
When a `ScalaUDF` dispatched into the native plan (#4514) participates in an expression that raises a divide-by-zero under ANSI mode (e.g. `1/udf(a)`), the error surfaces as `org.apache.comet.CometNativeException` ("DivideByZero") instead of the Spark `org.apache.spark.SparkArithmeticException` with error class `DIVIDE_BY_ZERO`. DataFusion wraps the typed error so the JNI bridge cannot downcast it back to the matching Spark exception.
This is observable in Spark's `SQLQueryTestSuite` golden test `udf/postgreSQL/udf-select_having.sql` (query `SELECT 1 AS one FROM test_having WHERE 1/udf(a) = 1 HAVING 1 < 2`). Same error category, different surface.
Follow-on from #4514. We should preserve the typed Spark exception (exception class plus error class) end to end so native evaluation matches Spark's ANSI error behavior.
Contributor guide
Assessment
This issue has not been assessed yet.