JuliaDataCubes / JuliaDataCubes/YAXArrays.jl
incorrect length of Time dim
Open
Nobody has claimed this yet.
- Dominant language
- Julia
- Stars
- 132
- Forks
- 25
- PR merge metrics
- No merged PRs in 30d
Description
using Zarr, YAXArrays
using DimensionalData
using CFTime
# chunksize(c) = Cubes.cubechunks(c)
store ="gs://cmip6/CMIP6/ScenarioMIP/DKRZ/MPI-ESM1-2-HR/ssp585/r1i1p1f1/3hr/tas/gn/v20190710/"
@time g = open_dataset(zopen(store, consolidated=true))
c = g.tas
@time kelvinCube = c[time=Between(DateTime("2030-01-01"), DateTime("2030-04-01"))]
@time savecube(kelvinCube, "temp-02.zarr")
julia> kelvinCube
384×192×721 YAXArray{Float32,3} with dimensions:
Dim{:lon} Sampled{Float64} 0.0:0.9375:359.0625 ForwardOrdered Regular Points,
Dim{:lat} Sampled{Float64} Float64[-89.28422753251364, -88.35700351866494, …, 88.35700351866494, 89.28422753251364] ForwardOrdered Irregular Points,
Ti Sampled{DateTime} DateTime[2030-01-01T00:00:00, …, 2030-04-01T00:00:00] ForwardOrdered Irregular Points
units: K
name: tas
Total size: 202.78 MB
Ti should has a length of 721.
However, the length of time dimension in the zarr file is 880
ds = Cube("temp-02.zarr") # correct if open by YAXArrays
ds.Time
julia> # length of time is 880
times = zopen("temp-02.zarr/Time")[:]
CFTime.timedecode(times, "days since 1980-01-01", "standard")
880-element Vector{DateTime}:
2029-12-12T03:00:00
2029-12-12T06:00:00
2029-12-12T09:00:00
2029-12-12T12:00:00
2029-12-12T15:00:00
2029-12-12T18:00:00
2029-12-12T21:00:00
2029-12-13T00:00:00
2029-12-13T03:00:00
2029-12-13T06:00:00
2029-12-13T09:00:00
2029-12-13T12:00:00
2029-12-13T15:00:00
⋮
2030-03-30T12:00:00
2030-03-30T15:00:00
2030-03-30T18:00:00
2030-03-30T21:00:00
2030-03-31T00:00:00
2030-03-31T03:00:00
2030-03-31T06:00:00
2030-03-31T09:00:00
2030-03-31T12:00:00
2030-03-31T15:00:00
2030-03-31T18:00:00
2030-03-31T21:00:00
2030-04-01T00:00:00
Double check in Python
import xarray as xr
ds = xr.open_dataset("./temp-02.zarr", engine = "zarr")
ds
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the Julia workflow using zopen, Cube, and savecube with the supplied temp-02.zarr example, then compare kelvinCube's 721 Time values with the 880 values in temp-02.zarr/Time. Check the corresponding Python xarray output as a second reference; done means the saved Zarr Time dimension matches the selected data and expected date range.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- julia
- Domain
- data
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 32/100