[Bug] pmod rejects string arguments that Spark implicitly casts to a numeric type
- Dominant language
- Rust
- Stars
- 9.3k
- Forks
- 2.4k
- Avg merge
- 3d 7h
- Merged PRs (30d)
- 344
Description
### Describe the bug
Spark's `Pmod` declares `inputType = NumericType` and relies on the analyzer's
string promotion rule, so a string argument is implicitly cast. DataFusion's
`SparkPmod` uses `Signature::numeric(2, Volatility::Immutable)`, which rejects
strings at planning time.
### To Reproduce
Spark 4.2.0 (verified with `pyspark==4.2.0`), and covered by Spark's own golden
file `sql/core/src/test/resources/sql-tests/results/typeCoercion/native/promoteStrings.sql.out`:
```sql
SELECT pmod('1', CAST(1 AS TINYINT)); -- 0, result type bigint
SELECT pmod('10', CAST(3 AS INT)); -- ansi on: 1 (int); ansi off: 1.0 (double)
SELECT pmod('1', '1'); -- DATATYPE_MISMATCH.BINARY_OP_WRONG_TYPE
```
Note that Spark's promotion target differs by mode: with
`spark.sql.ansi.enabled = true` the string is cast to the other argument's type,
and with it disabled the pair is promoted to `double`. Two strings are rejected
in both modes.
DataFusion:
```sql
SELECT pmod('10'::string, 3::int);
-- Error during planning: For function 'pmod' Utf8 and Int32 are not coercible
-- to a common numeric type.
```
### Expected behavior
A string argument is accepted and coerced the way Spark coerces it.
### Scope
This is not specific to `pmod`. Every `datafusion-spark` function that uses
`Signature::numeric` inherits it, and reproducing Spark's mode-dependent
promotion target needs a decision about how far `datafusion-spark` should go in
emulating Spark's analyzer. Filing it so the gap is visible rather than
proposing a `pmod`-local workaround.
A commented-out query recording the Spark 4.2.0 result is in
`datafusion/sqllogictest/test_files/spark/math/pmod.slt`.
### Relevant code
`datafusion/spark/src/function/math/modulus.rs`, `SparkPmod::new`.
Surfaced by the audit-datafusion-spark-expression skill.
Contributor guide
Assessment
This issue has not been assessed yet.