duckdb / duckdb/duckdb-spatial

Query performance is slower after using SPATIAL_JOIN

Open
#574 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
C
Stars
708
Forks
96
Avg merge
1d 21h
Merged PRs (30d)
5

Description

With the release of DuckDB v1.3.0, performance improvements for the spatial extension were announced. I ran some tests to evaluate this.
However, in my specific test scenario, I found that queries are actually faster when `SPATIAL_JOIN` is *not* used compared to when it *is* used.

❯ duckdb --version
v1.3.0 71c5c07cdd

❯ uname -a
Darwin mini.local 23.4.0 Darwin Kernel Version 23.4.0: Fri Mar 15 00:12:41 PDT 2024; root:xnu-10063.101.17~1/RELEASE_ARM64_T8103 arm64

Here is my test case:
`pre.shp` is an isoline dataset for precipitation values ranging from 0.1 to 10.
`grid.shp` is a point dataset.

Test files are attached. [data.zip](https://github.com/user-attachments/files/20389382/data.zip)

```

load spatial;
create table pre(
id integer primary key,
geom geometry
);
create sequence seq_pre_id start 1;
insert into pre(id,geom) select nextval('seq_pre_id'),geom from st_read('pre.shp');

create table grid(
id integer primary key,
geom geometry
);
create sequence seq_grid_id START 1;
insert into grid(id,geom) select nextval('seq_grid_id'),geom from st_read('grid.shp');

.timer on
select count(*) from
pre a,grid b where ST_Intersects(b.geom,a.geom);

┌──────────────┐
│ count_star() │
│ int64 │
├──────────────┤
│ 228909 │
└──────────────┘
Run Time (s): real 14.055 user 45.078702 sys 0.101822

explain
select count(*) from
pre a,grid b where ST_Intersects(b.geom,a.geom);

┌─────────────────────────────┐
│┌───────────────────────────┐│
││ Physical Plan ││
│└───────────────────────────┘│
└─────────────────────────────┘
┌───────────────────────────┐
│ UNGROUPED_AGGREGATE │
│ ──────────────────── │
│ Aggregates: │
│ count_star() │
└─────────────┬─────────────┘
┌─────────────┴─────────────┐
│ SPATIAL_JOIN │
│ ──────────────────── │
│ Join Type: INNER │
│ │
│ Conditions: ├──────────────┐
│ ST_Intersects(geom, geom) │ │
│ │ │
│ ~512260 Rows │ │
└─────────────┬─────────────┘ │
┌─────────────┴─────────────┐┌─────────────┴─────────────┐
│ SEQ_SCAN ││ SEQ_SCAN │
│ ──────────────────── ││ ──────────────────── │
│ Table: grid ││ Table: pre │
│ Type: Sequential Scan ││ Type: Sequential Scan │
│ Projections: geom ││ Projections: geom │
│ ││ │
│ ~512260 Rows ││ ~241 Rows │
└───────────────────────────┘└───────────────────────────┘
Run Time (s): real 0.005 user 0.001407 sys 0.001659

pragma disabled_optimizers = 'extension';

select count(*) from
pre a,grid b where ST_Intersects(b.geom,a.geom);

┌──────────────┐
│ count_star() │
│ int64 │
├──────────────┤
│ 228909 │
└──────────────┘
Run Time (s): real 10.304 user 55.053951 sys 0.247437

explain
select count(*) from
pre a,grid b where ST_Intersects(b.geom,a.geom);

┌─────────────────────────────┐
│┌───────────────────────────┐│
││ Physical Plan ││
│└───────────────────────────┘│
└─────────────────────────────┘
┌───────────────────────────┐
│ UNGROUPED_AGGREGATE │
│ ──────────────────── │
│ Aggregates: │
│ count_star() │
└─────────────┬─────────────┘
┌─────────────┴─────────────┐
│ BLOCKWISE_NL_JOIN │
│ ──────────────────── │
│ Join Type: INNER │
│ ├──────────────┐
│ Condition: │ │
│ ST_Intersects(geom, geom) │ │
└─────────────┬─────────────┘ │
┌─────────────┴─────────────┐┌─────────────┴─────────────┐
│ SEQ_SCAN ││ SEQ_SCAN │
│ ──────────────────── ││ ──────────────────── │
│ Table: grid ││ Table: pre │
│ Type: Sequential Scan ││ Type: Sequential Scan │
│ Projections: geom ││ Projections: geom │
│ ││ │
│ ~512260 Rows ││ ~241 Rows │
└───────────────────────────┘└───────────────────────────┘
Run Time (s): real 0.003 user 0.001087 sys 0.001004
```

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.