Window functions seem to be very slow
- Dominant language
- Rust
- Stars
- 503
- Forks
- 61
- Avg merge
- 2d 4h
- Merged PRs (30d)
- 90
Description
I was trying to port some code from duckdb to sedonadb, while geospatial functions were very fast window functions take significantly more time than expected. I tried running same query (in example below) 5 times on duckdb and sedonadb, with duckdb it took 5-8 seconds and with sedonadb it took 16-22 seconds.
## Reproducibe python code
```python
import sedona.db
import duckdb
sd = sedona.db.connect()
dcon = duckdb.connect()
query = """
WITH RECURSIVE numbers AS (
SELECT 1 AS id
UNION ALL
SELECT id + 1
FROM numbers
WHERE id < 100000
),
mock_data AS (
SELECT
id,
'Department_' || CAST((id % 10) AS CHAR) AS department,
ROUND(ABS(SIN(id)) * 500, 2) AS amount
FROM numbers
)
SELECT
id,
department,
amount,
ROW_NUMBER() OVER(PARTITION BY department ORDER BY amount DESC, id ASC) AS row_num,
COUNT(*) OVER(PARTITION BY department) AS dept_total_records
FROM mock_data
LIMIT 20;
"""
sd.sql(query).show() # 17, 18, 22, 22, 16
dcon.sql(query) # 5, 5, 8, 8, 7
```
## Version
sedonadb=0.4.0
python=3.14.3
duckdb=1.5.4
Contributor guide
Research direction
Start by running the reproducible Python query from the issue against SedonaDB 0.4.0 and DuckDB 1.5.4, recording the reported timings. Trace the window-function execution involved in ROW_NUMBER and COUNT over the partitioned mock data; done means the performance gap is explained and the window query is improved or the limitation is documented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, rust
- Domain
- databases, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100