apache / apache/sedona-db

Window functions seem to be very slow

Open
#1,266 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
503
Forks
61
Avg merge
2d 4h
Merged PRs (30d)
90

Description

I was trying to port some code from duckdb to sedonadb, while geospatial functions were very fast window functions take significantly more time than expected. I tried running same query (in example below) 5 times on duckdb and sedonadb, with duckdb it took 5-8 seconds and with sedonadb it took 16-22 seconds.

## Reproducibe python code
```python
import sedona.db
import duckdb

sd = sedona.db.connect()
dcon = duckdb.connect()

query = """

WITH RECURSIVE numbers AS (
SELECT 1 AS id
UNION ALL
SELECT id + 1
FROM numbers
WHERE id < 100000
),
mock_data AS (
SELECT
id,
'Department_' || CAST((id % 10) AS CHAR) AS department,
ROUND(ABS(SIN(id)) * 500, 2) AS amount
FROM numbers
)
SELECT
id,
department,
amount,

ROW_NUMBER() OVER(PARTITION BY department ORDER BY amount DESC, id ASC) AS row_num,

COUNT(*) OVER(PARTITION BY department) AS dept_total_records
FROM mock_data

LIMIT 20;

"""

sd.sql(query).show() # 17, 18, 22, 22, 16
dcon.sql(query) # 5, 5, 8, 8, 7

```

## Version
sedonadb=0.4.0
python=3.14.3
duckdb=1.5.4

Contributor guide

Open the contributing guide

Research direction

Start by running the reproducible Python query from the issue against SedonaDB 0.4.0 and DuckDB 1.5.4, recording the reported timings. Trace the window-function execution involved in ROW_NUMBER and COUNT over the partitioned mock data; done means the performance gap is explained and the window query is improved or the limitation is documented.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, rust
Domain
databases, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.