surface_distance(): the Dijkstra heap is sized H*W, so dense source rasters write out of bounds and abort the process
Nessuno ha ancora preso questa issue.
Valutazione
- Difficoltà
- 3/5
- Tempo stimato
- 1-2 giorni
- Idoneità per principianti
- 78/100
- Tipo di issue
- Bug
- Chiarezza
- Specificata chiaramente
- Stato di attività
- Tranquilla
- Stack tecnologico
- numpy, python
- Ambito
- backend, performance
Direzione di ricerca
Start in xrspatial/surface_distance.py at _dijkstra and _dijkstra_geodesic, then compare their heap allocation with the corrected sizing and memory comment in xrspatial/cost_distance.py. Reproduce the dense checkerboard case with NUMBA_BOUNDSCHECK=1 and verify both kernels and the _check_memory accounting no longer overflow.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Descrizione
Summary
_dijkstra and _dijkstra_geodesic in xrspatial/surface_distance.py size the
binary-heap backing arrays as
max_heap = height * width # lines 214 and 277
h_keys = np.empty(max_heap, dtype=np.float64)
h_rows = np.empty(max_heap, dtype=np.int64)
h_cols = np.empty(max_heap, dtype=np.int64)
Both kernels are lazy-deletion Dijkstra: a pixel is pushed onto the heap every
time its tentative distance improves, so it can be enqueued once per settled
neighbour rather than once overall. The push count is bounded by
height * width * (n_neighbors + 1), not height * width.
_heap_push (in cost_distance.py) writes keys[pos] at pos = size with no
bounds check. Once the heap outgrows the allocation, the kernel writes past the
end of three numba arrays. numba compiles without bounds checking by default, so
nothing catches it.
cost_distance.py carried the same defect and was already corrected; see the
comment at xrspatial/cost_distance.py:155-161 and the sizing at line 161.
surface_distance.py was never updated.
Reproduction
A checkerboard source raster over random elevation, 8-connectivity, on the
plain numpy backend:
import numpy as np
import xarray as xr
from xrspatial.surface_distance import surface_distance
rng = np.random.default_rng(0)
H = W = 80
elev = rng.random((H, W)) * 5000.0
rr, cc = np.meshgrid(np.arange(H), np.arange(W), indexing='ij')
src = ((rr + cc) % 2 == 0).astype(np.float64)
coords = {'y': np.arange(H, dtype=float), 'x': np.arange(W, dtype=float)}
raster = xr.DataArray(src, dims=('y', 'x'), coords=coords)
elevation = xr.DataArray(elev, dims=('y', 'x'), coords=coords)
surface_distance(raster, elevation, connectivity=8)
Default build (no bounds checking) — the process dies:
$ python repro.py
double free or corruption (!prev)
Aborted (core dumped)
Same script under NUMBA_BOUNDSCHECK=1, which names the defect:
$ NUMBA_BOUNDSCHECK=1 python repro.py
File "xrspatial/surface_distance.py", line 474, in _surface_distance_numpy
_dijkstra(elev_data, H, W, max_distance,
dy, dx, dd, dist, alloc, src_row, src_col)
IndexError: index is out of bounds
Affected backends
Reproduced under NUMBA_BOUNDSCHECK=1 with the same 80x80 input:
| Path | Result |
|---|---|
numpy, planar (_dijkstra) |
IndexError: index is out of bounds |
numpy, geodesic (_dijkstra_geodesic) |
IndexError: index is out of bounds |
dask+numpy, iterative tile path via _run_tile |
IndexError: index is out of bounds |
The dask+numpy bounded map_overlap branch runs _surface_distance_numpy per
chunk, so it is exposed on any chunk whose seeded pixel density is high enough.
The cupy and dask+cupy backends use iterative parallel relaxation with no heap
and are not affected. compute-sanitizer --tool memcheck over nine adversarial
shapes (2x2, 2x97, 97x2, 3x101, 101x3, 61x59, 127x127, 33x512, 512x33) across all
three public functions reports ERROR SUMMARY: 0 errors, and the
_sd_relax_kernel bounds guard at line 496 is correct.
How often it bites
Peak concurrent heap size against the allocated cap, measured with an
instrumented copy of _dijkstra (oversized heap so it can run to completion),
8-connectivity, random elevation:
| Raster | Seeding | Peak heap | Allocated cap | Overflow |
|---|---|---|---|---|
| 60x60 | single source | 452 | 3600 | no (0.13x) |
| 100x100 | 20% random sources | 9810 | 10000 | no (0.98x) |
| 40x40 | checkerboard | 1614 | 1600 | yes (1.01x) |
| 80x80 | checkerboard | 6646 | 6400 | yes (1.04x) |
| 200x200 | checkerboard | 41711 | 40000 | yes (1.04x) |
Sparse single-source rasters stay well clear, which is why the existing tests
pass. Dense source masks are the trigger, and those are ordinary usage:
a rasterized road or stream network, a landcover class mask, or any
target_values= selection that matches a large share of the raster. A 20%
random source density already reaches 0.98x of the cap.
Issue #3709 notes that the module's own benchmark raster makes 99.95% of pixels
sources, so the benchmark suite is already running in the regime that overflows.
Proposed fix
Size the heap to the actual bound in both kernels, matching what
cost_distance.py already does:
max_heap = height * width * (n_neighbors + 1)
The _BYTES_PER_PIXEL = 80 constant used by _check_memory models the heap at
24 bytes/pixel, so it needs to follow the new allocation or the guard added in
#1305 will under-count the eager numpy footprint by roughly 3.4x.
Related, not filed
xrspatial/pathfinding.py:304 uses the same max_heap = height * width sizing
with the same _heap_push. That kernel is single-source A*, and I could not get
it to overflow, so I am recording it here as a site to re-check rather than
claiming a defect.
Found by /deep-sweep security audit of surface_distance on 2026-08-16.
- Lingua principale
- Python
- Stelle
- 972
- Fork
- 92
- Merge medio
- 2g 12h
- PR unite (30g)
- 7
Guida per i contributori
Apri la guida per i contributori
Come iniziare
- Leggi tutta la issue e poi la guida ai contributi del progetto.
- Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
- Fai un fork del repository e lavora su un branch.
- Apri una pull request che faccia riferimento al numero della issue.
Altre issue di xarray-contrib/xarray-spatial
-
Difficoltà 1/5 Meno di un'ora Idoneità per principianti 68/100
xarray-contrib/xarray-spatial#3726 ·
-
api area:surface bug severity:medium sweep-api-consistency
Difficoltà 2/5 1-3 ore Idoneità per principianti 82/100
xarray-contrib/xarray-spatial#3712 ·
-
Difficoltà 1/5 Meno di un'ora Idoneità per principianti 88/100
xarray-contrib/xarray-spatial#3710 ·
-
benchmarks: cupy branch in get_xr_dataarray never raises NotImplementedError on non-GPU hosts Apertabug
Difficoltà 1/5 1-3 ore Idoneità per principianti 88/100
xarray-contrib/xarray-spatial#3707 ·
-
area:surface documentation user-guide-example
Difficoltà 2/5 1-3 ore Idoneità per principianti 72/100
xarray-contrib/xarray-spatial#3464 ·
Tutte le issue di xarray-contrib/xarray-spatial
Issue simili
-
bug
Difficoltà 2/5 1-3 ore Idoneità per principianti 86/100
zostera/django-bootstrap4#894 ·
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 78/100
use-agent-os/agent-os#3276 ·
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 88/100
zephyrproject-rtos/zephyr#119726 ·
-
area/auth bug comp/agent P3 platform/discord type/security
Difficoltà 2/5 1-3 ore Idoneità per principianti 88/100
NousResearch/hermes-agent#117848 ·
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 82/100
zilliztech/memsearch#759 ·