JuliaDataCubes / JuliaDataCubes/YAXArrays.jl
Can we get YAXArrays to automatically parse datetime variables
Nobody has claimed this yet.
- Dominant language
- Julia
- Stars
- 132
- Forks
- 25
- PR merge metrics
- No merged PRs in 30d
Description
It seems that YAXArrays does a great job at parsing DateTime dimensions but not DateTime variables. Here's an example that points to open data.
load packages and data
using YAXArrays
using Zarr
path = "https://its-live-data.s3.amazonaws.com/datacubes/v2-updated-october2024/S70E070/ITS_LIVE_vel_EPSG3031_G0120_X1250000_Y350000.zarr";
store = zopen(path);
ds = YAXArrays.open_dataset(store);
inspect mid_date dimension
ds.mid_date
julia> mid_date [DateTime("2007-07-21T02:40:09.758"), …, DateTime("2024-12-19T02:19:33.907")]
inspect acquisition_date_img1 variable
ds.acquisition_date_img1
┌ 22242-element YAXArray{Float64, 1} ┐
├────────────────────────────────────┴─────────────────────────────────────────────────────────────────── dims ┐
↓ mid_date Sampled{DateTime} [DateTime("2007-07-21T02:40:09.758"), …, DateTime("2024-12-19T02:19:33.907")] Unordered Irregular Points
├──────────────────────────────────────────────────────────────────────────────────────────────────── metadata ┤
Dict{String, Any} with 6 entries:
"units" => "days since 1970-01-01"
"calendar" => "proleptic_gregorian"
"name" => "acquisition_date_img1"
"standard_name" => "image1_acquition_date"
"description" => "acquisition date and time of image 1"
"_FillValue" => NaN
├─────────────────────────────────────────────────────────────────────────────────────────────── loaded lazily ┤
data size: 173.77 KB
Notice that acquisition_date_img1 has not been converted to DateTime
ds.acquisition_date_img1[1]
julia> 13499.111201768137
Is there a way that we could get YAXArrays to automatically parse DateTime variables?
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at the YAXArrays.open_dataset entry point and trace how CF metadata such as units and calendar is applied to dimensions versus variables. Reproduce the provided Zarr example and add a regression test for acquisition_date_img1 using its days-since-epoch metadata. Done means eligible datetime variables are exposed as DateTime values without breaking numeric variables.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- julia
- Domain
- data
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100