JuliaDataCubes / JuliaDataCubes/YAXArrays.jl
mapCube does not work for combination of Dataset and datacube
Nobody has claimed this yet.
- Dominant language
- Julia
- Stars
- 132
- Forks
- 25
- PR merge metrics
- No merged PRs in 30d
Description
When I want to combine a dataset with another cube in a mapCube call, I run into the dispatch of Tuples and I get the error below.
When I am using one of the data cubes in the data set instead of the whole data set, the mapCube works.
I would expect this mapCube call to do the respective computation for every subcube of the dataset, because all data cubes share the Timeaxis.
julia> freqsample
YAXArray Dataset
Dimensions:
Sample_5393 Axis with 3750 elements: 5393_1 5393_2 .. 5393_3749 5393_3750
Sample_28109 Axis with 5069 elements: 28109_1 28109_2 .. 28109_5068 28109_5069
Sample_27533 Axis with 5547 elements: 27533_1 27533_2 .. 27533_5546 27533_5547
Polarisation Axis with 2 elements: VH VV
Sample_7482 Axis with 2597 elements: 7482_1 7482_2 .. 7482_2596 7482_2597
Time Axis with 100 Elements from 2016-10-03T10:12:28 to 2020-02-09T10:12:44
Sample_6187 Axis with 5595 elements: 6187_1 6187_2 .. 6187_5594 6187_5595
Sample_28045 Axis with 14126 elements: 28045_1 28045_2 .. 28045_14125 28045_14126
Sample_7555 Axis with 22929 elements: 7555_1 7555_2 .. 7555_22928 7555_22929
Sample_23787 Axis with 3707 elements: 23787_1 23787_2 .. 23787_3706 23787_3707
Sample_27588 Axis with 6924 elements: 27588_1 27588_2 .. 27588_6923 27588_6924
Sample_10016 Axis with 4324 elements: 10016_1 10016_2 .. 10016_4323 10016_4324
Variables: ID_5393 ID_6187 ID_7482 ID_7555 ID_10016 ID_23787 ID_27533 ID_27588 ID_28045 ID_28109
julia> freqsamplenorm = mapCube((x,y)->x ./ y, (freqsample, orgmean), indims=(InDims(:Time), InDims(:Time)), outdims=OutDims(:Time), inplace=false)
ERROR: MethodError: no method matching ndims(::Dataset)
Closest candidates are:
ndims(::Union{HDF5.Attribute, HDF5.Dataset}) at ~/.julia/packages/HDF5/NqGY2/src/dataspaces.jl:257
ndims(::DimensionalData.Dimensions.Dimension{<:AbstractArray}) at ~/.julia/packages/DimensionalData/K9D4P/src/Dimensions/dimension.jl:253
ndims(::DimensionalData.Dimensions.Dimension) at ~/.julia/packages/DimensionalData/K9D4P/src/Dimensions/dimension.jl:252
...
Stacktrace:
[1] dimnames(x::Dataset)
@ YAXArrayBase ~/.julia/dev/YAXArrayBase/src/axisarrays/axisinterface.jl:46
[2] caxes(x::Dataset)
@ YAXArrays.Cubes ~/.julia/dev/YAXArrays/src/Cubes/Cubes.jl:171
[3] findAxis(desc::YAXArrays.Cubes.Axes.ByName, c::Dataset)
@ YAXArrays.Cubes.Axes ~/.julia/dev/YAXArrays/src/Cubes/Axes.jl:323
[4] _broadcast_getindex_evalf
@ ./broadcast.jl:670 [inlined]
[5] _broadcast_getindex
@ ./broadcast.jl:643 [inlined]
[6] (::Base.Broadcast.var"#29#30"{Base.Broadcast.Broadcasted{Base.Broadcast.Style{Tuple}, Nothing, typeof(YAXArrays.Cubes.Axes.findAxis), Tuple{Tuple{YAXArrays.Cubes.Axes.ByName}, Base.RefValue{Dataset}}}})(k::Int64)
@ Base.Broadcast ./broadcast.jl:1075
[7] ntuple
@ ./ntuple.jl:48 [inlined]
[8] copy(bc::Base.Broadcast.Broadcasted{Base.Broadcast.Style{Tuple}, Nothing, typeof(YAXArrays.Cubes.Axes.findAxis), Tuple{Tuple{YAXArrays.Cubes.Axes.ByName}, Base.RefValue{Dataset}}})
@ Base.Broadcast ./broadcast.jl:1075
[9] materialize
@ ./broadcast.jl:860 [inlined]
[10] YAXArrays.DAT.InputCube(c::Dataset, desc::InDims)
@ YAXArrays.DAT ~/.julia/dev/YAXArrays/src/DAT/DAT.jl:76
[11] (::YAXArrays.DAT.var"#30#33")(o::Tuple{Dataset, InDims})
@ YAXArrays.DAT ./none:0
[12] iterate
@ ./generator.jl:47 [inlined]
[13] collect(itr::Base.Generator{Base.Iterators.Zip{Tuple{Tuple{Dataset, YAXArray{Union{Missing, Float32}, 2, Matrix{Union{Missing, Float32}}, Vector{CubeAxis}}}, Tuple{InDims, InDims}}}, YAXArrays.DAT.var"#30#33"})
@ Base ./array.jl:787
[14] YAXArrays.DAT.DATConfig(cdata::Tuple{Dataset, YAXArray{Union{Missing, Float32}, 2, Matrix{Union{Missing, Float32}}, Vector{CubeAxis}}}, indims::Tuple{InDims, InDims}, outdims::OutDims, inplace::Bool, max_cache::Float64, fu::Function, ispar::Bool, include_loopvars::Bool, allow_irregular::Bool, nthreads::Vector{Int64}, addargs::Tuple{}, kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
@ YAXArrays.DAT ~/.julia/dev/YAXArrays/src/DAT/DAT.jl:266
[15] mapCube(::var"#43#44", ::Tuple{Dataset, YAXArray{Union{Missing, Float32}, 2, Matrix{Union{Missing, Float32}}, Vector{CubeAxis}}}; max_cache::Float64, indims::Tuple{InDims, InDims}, outdims::OutDims, inplace::Bool, ispar::Bool, debug::Bool, include_loopvars::Bool, showprog::Bool, irregular_loopranges::Bool, nthreads::Vector{Int64}, loopchunksize::Dict{Any, Any}, kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
@ YAXArrays.DAT ~/.julia/dev/YAXArrays/src/DAT/DAT.jl:434
[16] top-level scope
@ REPL[232]:1
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the mapCube call from the issue, then trace Dataset handling through YAXArrays/src/DAT/DAT.jl and the reported YAXArrayBase/src/axisarrays/axisinterface.jl, YAXArrays/src/Cubes/Cubes.jl, and YAXArrays/src/Cubes/Axes.jl locations. Done means mapCube accepts the Dataset alongside the other cube and performs the computation for each subcube sharing the Time axis.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- julia
- Domain
- data
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100