JuliaDataCubes / JuliaDataCubes/PyramidScheme.jl

Pyramid from Raster seems to be faster than from YAXArray

Open
#26 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Julia
Stars
18
Forks
3
PR merge metrics
No merged PRs in 30d

Description

When I am computing the Pyramid for a Raster this seems to be roughly a factor of two faster than computing the pyramid from a YAXArray with the same underlying data. This is with the example data "crs_enabled_zarr_and_tif/some.tif" from the geozarr interoperability issue. I am not sure where these differences come from and whether this is something that is worth looking into.

These are the timings I get for computing the pyramid for the first band.

julia> @time pyrc = PS.Pyramid(b1c)
  2.899440 seconds (44.07 M allocations: 1.908 GiB, 15.17% gc time)
╭──────────────────────────────╮
│ 8192×8192 Pyramid{Float32,2} │
├──────────────────────────────┴─────────────────────────────────────── dims ┐
  ↓ X Sampled{Float64} 400000.0:10.0:481910.0 ForwardOrdered Regular Points,
  → Y Sampled{Float64} 6.0e6:-10.0:5.91809e6 ReverseOrdered Regular Points
├────────────────────────────────────────────────────────────────────────────┴───────────────── metadata ┐
  Dict{String, Any} with 1 entry:
  "projection" => "PROJCS[\"WGS 84 / NSIDC EASE-Grid 2.0 Global\",GEOGCS[\"WGS 84\",DATUM[\"WGS_1984\",SP…
└────────────────────────────────────────────────────────────────────────────────────────────────────────┘
 ↓ →  6.0e6  5.99999e6  5.99998e6  5.99997e6  5.99996e6  …  5.91812e6  5.91811e6  5.9181e6  5.91809e6
 ⋮                                            ⋮          ⋱                                  

julia> @time pyr = PS.Pyramid(b1)
  1.303971 seconds (44.07 M allocations: 1.408 GiB, 4.77% gc time)
╭──────────────────────────────╮
│ 8192×8192 Pyramid{Float32,2} │
├──────────────────────────────┴─────────────────────────────────────────────────────────────────── dims ┐
  ↓ X Projected{Float64} LinRange{Float64}(400000.0, 481910.0, 8192) ForwardOrdered Regular Intervals{Start},
  → Y Projected{Float64} LinRange{Float64}(5.99999e6, 5.91808e6, 8192) ReverseOrdered Regular Intervals{Start}
├────────────────────────────────────────────────────────────────────────────────────────────── metadata ┤
  Metadata{Rasters.GDALsource} of Dict{String, Any} with 4 entries:
  "units"    => ""
  "offset"   => 0.0
  "filepath" => "test/data/crs_enabled_zarr_and_tif/some.tif"
  "scale"    => 1.0
└────────────────────────────────────────────────────────────────────────────────────────────────────────┘
 ⋮      ⋱  

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the Julia @time comparisons for PS.Pyramid(b1c) and PS.Pyramid(b1) using test/data/crs_enabled_zarr_and_tif/some.tif and the first-band examples in the issue. Trace the Pyramid entry points to identify the source of the timing difference; done should include a clear explanation and, if a change is warranted, benchmarked results.

Written by the indexing model from the issue text.

Assessment

Tech stack
julia
Domain
data, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.