JuliaDataCubes / JuliaDataCubes/PyramidScheme.jl
Pyramid from Raster seems to be faster than from YAXArray
Nobody has claimed this yet.
- Dominant language
- Julia
- Stars
- 18
- Forks
- 3
- PR merge metrics
- No merged PRs in 30d
Description
When I am computing the Pyramid for a Raster this seems to be roughly a factor of two faster than computing the pyramid from a YAXArray with the same underlying data. This is with the example data "crs_enabled_zarr_and_tif/some.tif" from the geozarr interoperability issue. I am not sure where these differences come from and whether this is something that is worth looking into.
These are the timings I get for computing the pyramid for the first band.
julia> @time pyrc = PS.Pyramid(b1c)
2.899440 seconds (44.07 M allocations: 1.908 GiB, 15.17% gc time)
╭──────────────────────────────╮
│ 8192×8192 Pyramid{Float32,2} │
├──────────────────────────────┴─────────────────────────────────────── dims ┐
↓ X Sampled{Float64} 400000.0:10.0:481910.0 ForwardOrdered Regular Points,
→ Y Sampled{Float64} 6.0e6:-10.0:5.91809e6 ReverseOrdered Regular Points
├────────────────────────────────────────────────────────────────────────────┴───────────────── metadata ┐
Dict{String, Any} with 1 entry:
"projection" => "PROJCS[\"WGS 84 / NSIDC EASE-Grid 2.0 Global\",GEOGCS[\"WGS 84\",DATUM[\"WGS_1984\",SP…
└────────────────────────────────────────────────────────────────────────────────────────────────────────┘
↓ → 6.0e6 5.99999e6 5.99998e6 5.99997e6 5.99996e6 … 5.91812e6 5.91811e6 5.9181e6 5.91809e6
⋮ ⋮ ⋱
julia> @time pyr = PS.Pyramid(b1)
1.303971 seconds (44.07 M allocations: 1.408 GiB, 4.77% gc time)
╭──────────────────────────────╮
│ 8192×8192 Pyramid{Float32,2} │
├──────────────────────────────┴─────────────────────────────────────────────────────────────────── dims ┐
↓ X Projected{Float64} LinRange{Float64}(400000.0, 481910.0, 8192) ForwardOrdered Regular Intervals{Start},
→ Y Projected{Float64} LinRange{Float64}(5.99999e6, 5.91808e6, 8192) ReverseOrdered Regular Intervals{Start}
├────────────────────────────────────────────────────────────────────────────────────────────── metadata ┤
Metadata{Rasters.GDALsource} of Dict{String, Any} with 4 entries:
"units" => ""
"offset" => 0.0
"filepath" => "test/data/crs_enabled_zarr_and_tif/some.tif"
"scale" => 1.0
└────────────────────────────────────────────────────────────────────────────────────────────────────────┘
⋮ ⋱
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the Julia @time comparisons for PS.Pyramid(b1c) and PS.Pyramid(b1) using test/data/crs_enabled_zarr_and_tif/some.tif and the first-band examples in the issue. Trace the Pyramid entry points to identify the source of the timing difference; done should include a clear explanation and, if a change is warranted, benchmarked results.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- julia
- Domain
- data, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100