CliMA / CliMA/CalibrateEmulateSample.jl
Blueprint for Dimension reducer / Data Processor
- Dominant language
- Julia
- Stars
- 90
- Forks
- 16
- PR merge metrics
- No merged PRs in 30d
Description
## Proposal
To construct a data processing tool that can unify the data processing of emulators.
Takes in
- Data: input-output pairs, or the EKI object (with i-o extracted internally)
- a schedule of DataProcessing & Dimension reduction
Gives back
- the processed data pairs
## Example
The user could specify `N`-stage processing:
In each stage the data can be processed with an:
```julia
process_schedule = [
("in", DataProcessor1(...)), # first process inputs with processor 1
("out", DataProcessor2(...)), # next process outputs with processor 2
("joint", DataProcessor3(...)), # next jointly process inputs and outputs with processor 3
("out", DataProcessor4(...)), # finally. process outputs with processor 4
]
```
The user then builds emulators on the processed data
```julia
io_pairs_processed = process_data(io_pairs, process_schedule)
# one catch: interfaces must handle that io_pairs_processed may be a different dimension to io_pairs
```
```julia
io_pairs_recovered= reverse_process_data(io_pairs, process_schedule)
# one catch: interfaces must handle that io_pairs_processed may be a different dimension to io_pairs
```
Where `DataProcessor(...)` could be selected from (just input or output)
- PCA
- simple scalings, or regularization
- Nonlinear dim reduction
or (joint input and output)
- data-informed or likelihood informed subspaces
- CCA
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.