duckdb / duckdb/duckdb-java

Optimizer incorrectly rewrites window plans involving arrow scan inputs to self-joins with multiple refs to the scan

Open
#713 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
127
Forks
80
Avg merge
13h 49m
Merged PRs (30d)
48

Description

This regressed in 1.5.0.

This affects queries that use window functions but _do not_ reference the arrow stream table more than once, e.g. `SELECT g, v, sum(v) OVER (PARTITION BY g) AS s FROM %s`.

1.4.4 always left this as one ARROW_SCAN_DUMB in the plan. 1.5.0 rewrites this to a self-join with multiple ARROW_SCAN_DUMB nodes, which doesn't work. Manifests as:

`java.sql.SQLException: Invalid Input Error: This stream has been released`

Related to: https://github.com/duckdb/duckdb-python/issues/70

I'm filing a separate issue because that preexisting issue describes a query that references the arrow stream multiple times. If that were the limitation, it's much clearer boundary: There are still situations where you can use arrow stream inputs and it's easy to understand when you can and when you can't (multiple references no, one reference yes). This new regression breaks that contract.

(PR: https://github.com/duckdb/duckdb/pull/23323)

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.