apache / apache/datafusion-comet
Handle remainder by zero error in `spark_modulo` function for Spark 4.1 and later.
- Dominant language
- Scala
- Stars
- 1.3k
- Forks
- 373
- Avg merge
- 2d 6h
- Merged PRs (30d)
- 190
Description
### What is the problem the feature request solves?
PR https://github.com/apache/datafusion-comet/pull/1971 introduced `spark_modulo` function, and found that Spark 4.0 and earlier version threw `Divide by Zero` instead of `Remainder by Zero` exception, which was incorrect. And so, filed https://issues.apache.org/jira/browse/SPARK-52659 for Spark to fix this.
That error message has now been fixed in the PR https://github.com/apache/spark/pull/51378 for Spark 4.1 and it will now throw `Reminder by Zero` instead of misleading `Divide by Zero`. This would require handling new error [here](https://github.com/apache/datafusion-comet/blob/fd0ab6441b078a8875b614ca2d615a102c592606/native/spark-expr/src/math_funcs/modulo_expr.rs#L58) for spark version 4.1 and later.
### Describe the potential solution
_No response_
### Additional context
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.