RS_EnsureContiguous UDF: materialize strided band views to contiguous bytes via an explicit plan node
- Dominant language
- Rust
- Stars
- 503
- Forks
- 61
- Avg merge
- 2d 4h
- Merged PRs (30d)
- 90
Description
## Problem
Non-identity band views (slice-with-step, broadcast, transpose, inner-axis slice) produce a *strided* byte layout that isn't packed row-major, so kernels needing packed bytes can't consume them directly. #813 establishes that the contiguous accessor (`NdBuffer::as_contiguous()`) returns bytes zero-copy when the layout is already packed and **errors** otherwise — it never silently materializes — and the GDAL bridge rejects strided views. Materialization is intentionally kept *out* of the read path: it belongs in an explicit, CSE-dedupable plan node. This issue adds that node.
## Proposal
Add `RS_EnsureContiguous`, mirroring `RS_EnsureLoaded` (#886):
- A UDF that takes a raster and returns one whose bands are identity-view, packed row-major — strided views materialized, already-contiguous bands passed through zero-copy.
- A logical optimizer rule (in `sedona-query-planner`) that wraps the raster args of kernels annotated `needs_contiguous`, using the `SedonaScalarUDF` metadata map (the `NEEDS_PIXELS_METADATA_KEY` mechanism generalizes to a `needs_contiguous` key).
- The materialization primitive uses caller-managed scratch (or a visitor) reused across rows — e.g. `NdBuffer::materialize_into(&mut Vec)` — never a per-row allocation. This is where the review thread on `array.rs:288` (externally-managed scratch / visitor) is addressed.
- Fast path: `NdBuffer::is_contiguous()` (added in #813) makes `RS_EnsureContiguous` a zero-copy no-op for rows whose layout is already packed — including contiguous *non-identity* views like outer-axis slices, not just the strict identity view.
## Composition with `RS_EnsureLoaded`
`RS_EnsureContiguous` needs bytes present, so it must run *after* load: `RS_X(RS_EnsureContiguous(RS_EnsureLoaded(rast)))`. Prefer unifying both into one metadata-driven normalization rule that injects the correct nested wrappers based on a kernel's flags, rather than two independent rules.
## Knock-on cleanups
- The GDAL bridge's strided-view rejection (#813) can relax once the plan guarantees identity-view inputs.
- Re-introduce the strided-copy helper removed in #813 (`materialize_strided`) here, as the scratch/visitor primitive.
## Design tenet
Materialization is never a transparent side effect of a trait accessor — the `BandRef` byte surface is `nd_buffer()` (zero-copy) plus `NdBuffer::as_contiguous()` (borrow-or-error). This UDF is the explicit, plan-visible place where strided→packed copying happens.
## Related
- #886 (`RS_EnsureLoaded` — the pattern this mirrors)
- #897 (`RS_EnsureLoaded` view round-trip — `EnsureLoaded` preserves the view; `EnsureContiguous` collapses it)
- #813 (view machinery — adds `is_contiguous`/`as_contiguous`, rejects strided reads)
Contributor guide
Research direction
Start by reading #886 for the RS_EnsureLoaded pattern, then inspect the planner in sedona-query-planner and the scratch/visitor discussion around array.rs:288. Compare the view and contiguous-buffer behavior from #813 and #897. Done means the explicit RS_EnsureContiguous UDF and metadata-driven normalization handle loaded strided inputs, reuse scratch across rows, and preserve zero-copy behavior for contiguous bands.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- backend-api-design, databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100