apache / apache/sedona-db

WKB import rejects LargeBinary storage

Open
#1,215 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
503
Forks
61
Avg merge
2d 4h
Merged PRs (30d)
90

Description

## Summary

Importing WKB whose Arrow storage is `LargeBinary` fails:

```
SedonaDB internal error: Expected Wkb type with Binary storage but got LargeBinary.
```

Reproducer through a literal (the same applies to any LargeBinary-backed `geoarrow.wkb` input):

```python
import geoarrow.pyarrow as ga, pyarrow as pa
from shapely.geometry import Point
from sedonadb.expr import lit

df.select(lit(pa.array([Point(1, 1).wkb], ga.large_wkb())))
```

`ga.large_wkb()` is a public GeoArrow type, and sources such as Iceberg return BLOB columns as LargeBinary, so this shows up with real data, not only constructed literals.

## Expected

WKB with `LargeBinary` storage is accepted wherever `Binary`-backed WKB is, keeping the CRS and edge-type metadata of the extension type.

## Related

- #1195 works around this on the Python side by rebuilding the type on `Binary` storage (with the same CRS and edge type) before resolving the literal; that can go once the importer accepts LargeBinary.

Contributor guide

Open the contributing guide

Research direction

Start by running the Python reproducer through the lit entry point and tracing the WKB importer error for LargeBinary storage. Confirm that LargeBinary-backed WKB is accepted while preserving CRS and edge-type metadata, then verify that the workaround described in #1195 is no longer needed.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, rust
Domain
databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.