apache / apache/arrow

[C++][Python] Substrait plan execution with invalid backpressure options causes erroneous exit.

Open
#33,375 0 comments 0 reactions 0 assignees View on GitHub
Type: bug
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 13h
Merged PRs (30d)
88

Description

ARROW-18025 is a work in progress to expose Backpressure handling to Python API. While working on options, found out that the python program doesn't exit or return an error when invalid backpressure options are passed when the output is not a RecordBatchReader or the PyTest raise is used to validate expected errors. 

In addition to this, if the `output_type` is set to `Table`, the observations is the same.

Seems like this is out of scope of the ARROW-18025 effort and creating this ticket to tackle the problem. 

Way to reproduce, 

Create a test case with invalid Backpressure options and set `output_type=Table`

Edited:

Make sure to remove the Python validation logic (backpressure validation) in `_substrait.pyx`.

**Reporter**: [Vibhatha Lakmal Abeykoon](https://issues.apache.org/jira/browse/ARROW-18187) / @vibhatha
**Assignee**: [Vibhatha Lakmal Abeykoon](https://issues.apache.org/jira/browse/ARROW-18187) / @vibhatha

**Note**: *This issue was originally created as [ARROW-18187](https://issues.apache.org/jira/browse/ARROW-18187). Please see the [migration documentation](https://github.com/apache/arrow/issues/14542) for further details.*

Contributor guide

Open the contributing guide

Research direction

Start with the Python binding code in `_substrait.pyx` and trace execution for invalid backpressure options when `output_type=Table` or the output is not a RecordBatchReader. Add a regression test using invalid options; done means the Python program returns or raises the expected error instead of exiting erroneously.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, python
Domain
api, backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.