JuliaDataCubes / JuliaDataCubes/YAXArrays.jl
Cube of dataset with a 0-dimensional subcube gives confusing error message
Nobody has claimed this yet.
- Dominant language
- Julia
- Stars
- 132
- Forks
- 25
- PR merge metrics
- No merged PRs in 30d
Description
I was trying to write up an example for the JuliaClimate meta issue about julia packages for the CMIP6 data
https://github.com/JuliaClimate/meta/issues/17 and I ran into the following error with trying to convert a CMIP6 dataset into a YAXArray Cube.
The problem is, that the dataset I randomly selected seem to have a zero dimensional subcube which can't be used to derive the size of the expected data cube from the concatenation.
julia> using AWS, Zarr, YAXArrays, CSV, DataFrames
julia> AWS.global_aws_config(AWSConfig(creds=nothing, region=""))
AWSConfig(nothing, "", "json")
julia> # table with all the URLs to the data on google cloud
CMIP6_stores = CSV.read(download("https://storage.googleapis.com/cmip6/cmip6-zarr-consolidated-stores.csv"), DataFrame)
julia> store = filter(CMIP6_stores) do row
row.activity_id == "ScenarioMIP" && row.institution_id=="DKRZ" && row.variable_id=="tas" && row.experiment_id=="ssp585"
end
3×11 DataFrame
Row │ activity_id institution_id source_id experiment_id member_id table_id variable_id grid_label zstore dcpp_init_year version
│ String15 String31 String31 String31 String15 String15 String31 String7 String Float64? Int64
─────┼──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
1 │ ScenarioMIP DKRZ MPI-ESM1-2-HR ssp585 r1i1p1f1 day tas gn gs://cmip6/CMIP6/ScenarioMIP/DKR… missing 20190710
2 │ ScenarioMIP DKRZ MPI-ESM1-2-HR ssp585 r1i1p1f1 Amon tas gn gs://cmip6/CMIP6/ScenarioMIP/DKR… missing 20190710
3 │ ScenarioMIP DKRZ MPI-ESM1-2-HR ssp585 r1i1p1f1 3hr tas gn gs://cmip6/CMIP6/ScenarioMIP/DKR… missing 20190710
julia> zopen(store.
activity_id dcpp_init_year experiment_id grid_label institution_id member_id source_id table_id variable_id version zstore
julia> zopen(store.zstore[1])
ZarrGroup at GCStore("cmip6") and path CMIP6/ScenarioMIP/DKRZ/MPI-ESM1-2-HR/ssp585/r1i1p1f1/day/tas/gn/v20190710/
Variables: lat height time tas lat_bnds lon_bnds lon time_bnds
julia> zopen(store.zstore[1])
ZarrGroup at GCStore("cmip6") and path CMIP6/ScenarioMIP/DKRZ/MPI-ESM1-2-HR/ssp585/r1i1p1f1/day/tas/gn/v20190710/
Variables: lat height time tas lat_bnds lon_bnds lon time_bnds
julia> @time zopen(store.zstore[1])
1.190920 seconds (6.51 k allocations: 692.555 KiB)
ZarrGroup at GCStore("cmip6") and path CMIP6/ScenarioMIP/DKRZ/MPI-ESM1-2-HR/ssp585/r1i1p1f1/day/tas/gn/v20190710/
Variables: lat height time tas lat_bnds lon_bnds lon time_bnds
julia> c = Cube(store.zstore[1])
ERROR: ArgumentError: invalid index: nothing of type Nothing
Stacktrace:
[1] to_index(i::Nothing)
@ Base ./indices.jl:300
[2] to_index(A::Vector{Any}, i::Nothing)
@ Base ./indices.jl:277
[3] to_indices
@ ./indices.jl:333 [inlined]
[4] to_indices
@ ./indices.jl:325 [inlined]
[5] getindex
@ ./abstractarray.jl:1241 [inlined]
[6] concatenatecubes(cl::Vector{Any}, cataxis::CategoricalAxis{String, :Variable, Vector{String}})
@ YAXArrays.Cubes ~/.julia/packages/YAXArrays/Fe7F8/src/Cubes/TransformedCubes.jl:31
[7] Cube(ds::Dataset; joinname::String)
@ YAXArrays.Datasets ~/.julia/packages/YAXArrays/Fe7F8/src/DatasetAPI/Datasets.jl:324
[8] Cube
@ ~/.julia/packages/YAXArrays/Fe7F8/src/DatasetAPI/Datasets.jl:299 [inlined]
[9] #Cube#122
@ ~/.julia/packages/YAXArrays/Fe7F8/src/DatasetAPI/Datasets.jl:744 [inlined]
[10] Cube(s::String)
@ YAXArrays.Datasets ~/.julia/packages/YAXArrays/Fe7F8/src/DatasetAPI/Datasets.jl:744
[11] top-level scope
@ REPL[15]:1
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the failure with the CMIP6 dataset example, then inspect concatenatecubes in src/Cubes/TransformedCubes.jl and Cube in src/DatasetAPI/Datasets.jl, as shown in the stack trace. Determine how the zero-dimensional subcube reaches the invalid index operation. Done means the input produces a clear error explaining the unsupported zero-dimensional subcube instead of invalid index: nothing.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- julia
- Domain
- data
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100