CliMA / CliMA/ClimaAnalysis.jl

Add support for reduction using dimension values in split-apply-combine pattern

Open
#399 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Julia
Stars
12
Forks
5
Avg merge
6d 17h
Merged PRs (30d)
3

Description

In the split-apply-combine, reductions are supported by `Reduction`. It would be useful to also pass information about the dimension values to the reduction. The syntax might looks like

```julia
time_averaged_var =
var |>
ClimaAnalysis.GroupAll("time") |>
ClimaAnalysis.ReduceWithDims(integrate) |>
ClimaAnalysis.combine
```

and the signature of the reduction could be `function integrate(data, dim_name, dims_values; dims) end` where `data` is being reduced over, `dim_name` is the name of the dimension, `dims_values` are all the dimension values (in the form of a dictionary mapping dimension name to dimension values), and `dims` is the index to do the reduction on.

The use case for this is to support operations like finding the argmax/argmin or integrating time series data.

One concern I have is whether it makes sense to pass in more information like the attributes and dimension attributes. Although, there are work around for this by defining a functor for the reduction that store the attributes and dimension attributes in the `OutputVar`s.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by tracing the existing split-apply-combine flow from GroupAll through Reduction and combine, then compare it with the proposed ReduceWithDims signature. Check how dimension names and values are currently represented and whether related tests cover reductions. Done means a reduction can receive the dimension values needed for operations such as argmax or time-series integration, with the API behavior documented by tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
julia
Domain
data
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.