r/sedonadb: `sd_read_sf()` crashes in filtering test
- Dominant language
- Rust
- Stars
- 503
- Forks
- 61
- Avg merge
- 2d 4h
- Merged PRs (30d)
- 90
Description
In CI on #1029 we see
```
Start test: sd_read_sf() works with filter
*** caught segfault ***
address 0x40, cause 'invalid permissions'
Traceback:
1: nanoarrow::convert_array_stream(stream, size = size)
2: st_as_sf.sedonadb_dataframe(sd_read_sf(nc_gpkg, filter = filter))
3: sf::st_as_sf(sd_read_sf(nc_gpkg, filter = filter))
4: eval(code, test_env)
5: eval(code, test_env)
```
https://github.com/apache/sedona-db/actions/runs/28835218309/job/85517449642?pr=1029#step:12:326
I am guessing that the `filter()` part triggers this (perhaps ephemerally) because it triggers a repartition, or perhaps a GDAL update. It would be a good idea to replace our GDAL-via-sf with something from sedona-gdal anyway, which may also help with #924 (also related to parallel GDAL OGR reads).
Contributor guide
Research direction
Start by reproducing the CI test “sd_read_sf() works with filter” from the #1029 run and follow the nanoarrow::convert_array_stream() stack trace through st_as_sf.sedonadb_dataframe(). Compare the filtered and unfiltered reads, including the possible repartition or GDAL path; done means the filtered read converts to sf without a segfault.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- r, rust
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100