JuliaDataCubes / JuliaDataCubes/YAXArrays.jl

Out of bounds indexing does not throw `BoundsError`?

Open
#426 1 comment 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

question
Dominant language
Julia
Stars
132
Forks
25
PR merge metrics
No merged PRs in 30d

Description

I accidentally indexed an 800x800x163 ish Zarr based YAXArray (pure Julia) by 1:1000, 1:1000, 1:15. This didn't throw a BoundsError as expected - instead, the operation went on silently, and when it came time to materialize the array threw the following error:


ERROR: ArgumentError: The array ZArray{Int16} of size 833 x 833 x 119 got missing chunks and no fill_value
Stacktrace:
  [1] uncompress_raw!(a::Array{…}, z::ZArray{…}, curchunk::Nothing)
    @ Zarr ~/.julia/dev/kerchunk-project/Zarr/src/ZArray.jl:262
  [2] uncompress_to_output!(aout::Array{…}, output_base_offsets::Tuple{…}, z::ZArray{…}, chunk_compressed::Nothing, current_chunk_offsets::Tuple{…}, a::Array{…}, indranges::Tuple{…})
    @ Zarr ~/.julia/dev/kerchunk-project/Zarr/src/ZArray.jl:275
  [3] readblock!(aout::Array{…}, z::ZArray{…}, r::CartesianIndices{…})
    @ Zarr ~/.julia/dev/kerchunk-project/Zarr/src/ZArray.jl:180
  [4] readblock!(::ZArray{…}, ::Array{…}, ::UnitRange{…}, ::Vararg{…})
    @ Zarr ~/.julia/dev/kerchunk-project/Zarr/src/ZArray.jl:249
  [5] getindex_disk_nobatch!(out::Nothing, a::ZArray{…}, i::Tuple{…})
    @ DiskArrays ~/.julia/packages/DiskArrays/6JA8Z/src/diskarray.jl:236
  [6] getindex_disk!
    @ ~/.julia/packages/DiskArrays/6JA8Z/src/diskarray.jl:252 [inlined]
  [7] getindex_disk
    @ ~/.julia/packages/DiskArrays/6JA8Z/src/diskarray.jl:210 [inlined]
  [8] getindex(::ZArray{…}, ::UnitRange{…}, ::UnitRange{…}, ::UnitRange{…})
    @ DiskArrays ~/.julia/packages/DiskArrays/6JA8Z/src/diskarray.jl:348
  [9] readblock!(::DiskArrayTools.CFDiskArray{…}, ::Array{…}, ::UnitRange{…}, ::Vararg{…})
    @ DiskArrayTools ~/.julia/packages/DiskArrayTools/141OI/src/DiskArrayTools.jl:268
 [10] getindex_disk_nobatch!(out::Nothing, a::DiskArrayTools.CFDiskArray{…}, i::Tuple{…})
    @ DiskArrays ~/.julia/packages/DiskArrays/6JA8Z/src/diskarray.jl:236
 [11] getindex_disk!
    @ ~/.julia/packages/DiskArrays/6JA8Z/src/diskarray.jl:252 [inlined]
 [12] getindex_disk
    @ ~/.julia/packages/DiskArrays/6JA8Z/src/diskarray.jl:210 [inlined]
 [13] getindex
    @ ~/.julia/packages/DiskArrays/6JA8Z/src/diskarray.jl:348 [inlined]
 [14] getindex(A::YAXArray{…}, i1::UnitRange{…}, i2::UnitRange{…}, Is::UnitRange{…})
    @ DimensionalData ~/.julia/packages/DimensionalData/RxCda/src/array/indexing.jl:130
 [15] top-level scope
    @ ~/.julia/packages/BenchmarkTools/QNsku/src/execution.jl:496
Some type information was truncated. Use `show(err)` to see complete types.

Not sure what the best solution is, I'll push an MWE tomorrow.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the issue with the promised minimal working example, using the out-of-bounds ranges against the 800x800x163-ish YAXArray. Start at DimensionalData's indexing.jl getindex entry point and follow the DiskArrays and Zarr stack frames; done means invalid ranges raise BoundsError before materialization rather than producing the reported missing-chunks error.

Written by the indexing model from the issue text.

Assessment

Tech stack
julia
Domain
data
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.