[C++][Python] Substrait plan execution with invalid backpressure options causes erroneous exit.
- Dominant language
- C++
- Stars
- 17.1k
- Forks
- 4.3k
- Avg merge
- 3d 13h
- Merged PRs (30d)
- 88
Description
ARROW-18025 is a work in progress to expose Backpressure handling to Python API. While working on options, found out that the python program doesn't exit or return an error when invalid backpressure options are passed when the output is not a RecordBatchReader or the PyTest raise is used to validate expected errors.
In addition to this, if the `output_type` is set to `Table`, the observations is the same.
Seems like this is out of scope of the ARROW-18025 effort and creating this ticket to tackle the problem.
Way to reproduce,
Create a test case with invalid Backpressure options and set `output_type=Table`
Edited:
Make sure to remove the Python validation logic (backpressure validation) in `_substrait.pyx`.
**Reporter**: [Vibhatha Lakmal Abeykoon](https://issues.apache.org/jira/browse/ARROW-18187) / @vibhatha
**Assignee**: [Vibhatha Lakmal Abeykoon](https://issues.apache.org/jira/browse/ARROW-18187) / @vibhatha
**Note**: *This issue was originally created as [ARROW-18187](https://issues.apache.org/jira/browse/ARROW-18187). Please see the [migration documentation](https://github.com/apache/arrow/issues/14542) for further details.*
Contributor guide
Research direction
Start with the Python binding code in `_substrait.pyx` and trace execution for invalid backpressure options when `output_type=Table` or the output is not a RecordBatchReader. Add a regression test using invalid options; done means the Python program returns or raises the expected error instead of exiting erroneously.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, python
- Domain
- api, backend
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100