apache / apache/sedona-db

r/sedonadb: `sd_read_sf()` crashes in filtering test

Open
#1,031 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
503
Forks
61
Avg merge
2d 4h
Merged PRs (30d)
90

Description

In CI on #1029 we see

```
Start test: sd_read_sf() works with filter

*** caught segfault ***
address 0x40, cause 'invalid permissions'

Traceback:
1: nanoarrow::convert_array_stream(stream, size = size)
2: st_as_sf.sedonadb_dataframe(sd_read_sf(nc_gpkg, filter = filter))
3: sf::st_as_sf(sd_read_sf(nc_gpkg, filter = filter))
4: eval(code, test_env)
5: eval(code, test_env)
```

https://github.com/apache/sedona-db/actions/runs/28835218309/job/85517449642?pr=1029#step:12:326

I am guessing that the `filter()` part triggers this (perhaps ephemerally) because it triggers a repartition, or perhaps a GDAL update. It would be a good idea to replace our GDAL-via-sf with something from sedona-gdal anyway, which may also help with #924 (also related to parallel GDAL OGR reads).

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the CI test “sd_read_sf() works with filter” from the #1029 run and follow the nanoarrow::convert_array_stream() stack trace through st_as_sf.sedonadb_dataframe(). Compare the filtered and unfiltered reads, including the possible repartition or GDAL path; done means the filtered read converts to sf without a segfault.

Written by the indexing model from the issue text.

Assessment

Tech stack
r, rust
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.