CliMA / CliMA/ClimaAnalysis.jl
Add support for reduction using dimension values in split-apply-combine pattern
- Dominant language
- Julia
- Stars
- 12
- Forks
- 5
- Avg merge
- 6d 17h
- Merged PRs (30d)
- 3
Description
In the split-apply-combine, reductions are supported by `Reduction`. It would be useful to also pass information about the dimension values to the reduction. The syntax might looks like
```julia
time_averaged_var =
var |>
ClimaAnalysis.GroupAll("time") |>
ClimaAnalysis.ReduceWithDims(integrate) |>
ClimaAnalysis.combine
```
and the signature of the reduction could be `function integrate(data, dim_name, dims_values; dims) end` where `data` is being reduced over, `dim_name` is the name of the dimension, `dims_values` are all the dimension values (in the form of a dictionary mapping dimension name to dimension values), and `dims` is the index to do the reduction on.
The use case for this is to support operations like finding the argmax/argmin or integrating time series data.
One concern I have is whether it makes sense to pass in more information like the attributes and dimension attributes. Although, there are work around for this by defining a functor for the reduction that store the attributes and dimension attributes in the `OutputVar`s.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by tracing the existing split-apply-combine flow from GroupAll through Reduction and combine, then compare it with the proposed ReduceWithDims signature. Check how dimension names and values are currently represented and whether related tests cover reductions. Done means a reduction can receive the dimension values needed for operations such as argmax or time-series integration, with the API behavior documented by tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- julia
- Domain
- data
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100