support `regexp_match` as predicate
- Dominant language
- Rust
- Stars
- 9.3k
- Forks
- 2.4k
- Avg merge
- 3d 7h
- Merged PRs (30d)
- 344
Description
### Describe the bug
From https://github.com/openobserve/openobserve/discussions/6584
regexp_match does not seem to work with length or space matches. see below.
### To Reproduce
```
$ mkdir ~/.cargo/tmp; TMPDIR=~/.cargo/tmp cargo install datafusion-cli
$ datafusion-cli
DataFusion CLI v47.0.0
> select * from 'test-datafusion.csv';
+---+-------+
| a | b |
+---+-------+
| 1 | one |
| 2 | two |
| 3 | three |
+---+-------+
3 row(s) fetched.
Elapsed 0.006 seconds.
> select * from 'test-datafusion.csv' where regexp_match(a, '(.){4,}');
Error during planning: Cannot create filter with non-boolean predicate 'regexp_match(test-datafusion.csv.a, Utf8("(.){4,}"))' returning List(Field { name: "item", data_type: Utf8, nullable: true, dict_id: 0, dict_is_ordered: false, metadata: {} })
> select * from 'test-datafusion.csv';
+---+--------------+
| a | b |
+---+--------------+
| 1 | one |
| 2 | two |
| 3 | three |
| 3 | four -O test |
+---+--------------+
4 row(s) fetched.
Elapsed 0.010 seconds.
> select * from 'test-datafusion.csv' where regexp_match(b, '(.){4,}');
Error during planning: Cannot create filter with non-boolean predicate 'regexp_match(test-datafusion.csv.b, Utf8("(.){4,}"))' returning List(Field { name: "item", data_type: Utf8, nullable: true, dict_id: 0, dict_is_ordered: false, metadata: {} })
> select * from 'test-datafusion.csv' where regexp_match(b, '\s-O\s');
Error during planning: Cannot create filter with non-boolean predicate 'regexp_match(test-datafusion.csv.b, Utf8("\s-O\s"))' returning List(Field { name: "item", data_type: Utf8, nullable: true, dict_id: 0, dict_is_ordered: false, metadata: {} })
```
### Expected behavior
both should return one result with no error.
### Additional context
_No response_
Contributor guide
Research direction
Start by reproducing the issue with datafusion-cli and the regexp_match predicates shown in the report, including the length and whitespace patterns. Trace how the predicate is planned and verify that both queries return one result without a non-boolean predicate error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust, sql
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100