[C++] Substrait: add mappings for starts_with, ends_with and match_substring
- Dominant language
- C++
- Stars
- 17.1k
- Forks
- 4.3k
- Avg merge
- 3d 13h
- Merged PRs (30d)
- 88
Description
`starts_with`, `ends_with` and `match_substring` have no Substrait mapping, so expressions using them cannot be serialized.
```python
import pyarrow as pa
import pyarrow.compute as pc
from pyarrow.substrait import serialize_expressions
schema = pa.schema([pa.field("cat", pa.string())])
serialize_expressions([pc.starts_with(pc.field("cat"), "al")], ["f"], schema)
```
```
pyarrow.lib.ArrowNotImplementedError: No conversion function exists to convert
the Arrow function starts_with to a Substrait call
```
Same for `pc.ends_with` and `pc.match_substring`. Comparisons, `isin` and arithmetic all serialize fine, so this looks like a per-function gap rather than a structural one.
Substrait defines `starts_with`, `ends_with` and `contains` in `functions_string.yaml`, which cover all three.
The signatures differ: the Substrait functions take the pattern as a second argument, while the Arrow kernels are unary and carry it in `MatchSubstringOptions`. So the mapping needs the second argument to be a string literal. Substrait's `case_sensitivity` option maps onto `ignore_case`, except for `CASE_INSENSITIVE_ASCII` which has no Arrow equivalent.
This matters for engines that ingest a PyArrow filter through Substrait, where an unmappable function turns into a hard failure rather than a fallback.
### Component(s)
C++, Python
Contributor guide
Assessment
This issue has not been assessed yet.