[Python] Add is_nan, is_null, is_valid as operators to DNF filters
Open
Component: Python
Type: enhancement
- Dominant language
- C++
- Stars
- 17.1k
- Forks
- 4.3k
- Avg merge
- 3d 13h
- Merged PRs (30d)
- 88
Description
### Describe the enhancement requested
Currently pyarrow.parquet.core.filters_to_expression handles equality/inequality operators and in and not in operators.
I propose adding is_nan, is_null, is_valid operators where value passed would be ignored but they would
return field.is_nan() field.is_null() field.is_valid() expressions.
This is a very easy change but it would allow null/nan filtering in DNF form.
These functions are already implemented for pyarrow.dataset.Expression https://arrow.apache.org/docs/python/generated/pyarrow.dataset.Expression.html
### Component(s)
Python
Contributor guide
Assessment
This issue has not been assessed yet.