Optimizer incorrectly rewrites window plans involving arrow scan inputs to self-joins with multiple refs to the scan
- Dominant language
- C++
- Stars
- 127
- Forks
- 80
- Avg merge
- 13h 49m
- Merged PRs (30d)
- 48
Description
This regressed in 1.5.0.
This affects queries that use window functions but _do not_ reference the arrow stream table more than once, e.g. `SELECT g, v, sum(v) OVER (PARTITION BY g) AS s FROM %s`.
1.4.4 always left this as one ARROW_SCAN_DUMB in the plan. 1.5.0 rewrites this to a self-join with multiple ARROW_SCAN_DUMB nodes, which doesn't work. Manifests as:
`java.sql.SQLException: Invalid Input Error: This stream has been released`
Related to: https://github.com/duckdb/duckdb-python/issues/70
I'm filing a separate issue because that preexisting issue describes a query that references the arrow stream multiple times. If that were the limitation, it's much clearer boundary: There are still situations where you can use arrow stream inputs and it's easy to understand when you can and when you can't (multiple references no, one reference yes). This new regression breaks that contract.
(PR: https://github.com/duckdb/duckdb/pull/23323)
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.