equinor / equinor/graphite-maps

Scale data before precision estimation

Open
#39 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
8
Forks
3
PR merge metrics
No merged PRs in 30d

Description

There can be a large difference between
```python
Prec_u_sub = fit_precision_cholesky(X, graph_u_sub, verbose_level=5)
```

and
```python
from sklearn.preprocessing import StandardScaler
scaler = StandardScaler()

# Fit the scaler to the data and transform it
X_scaled = scaler.fit_transform(X)
X_scaled.shape
Prec_u_sub = fit_precision_cholesky(X_scaled, graph_u_sub, verbose_level=5)
```

In particular, this is experienced on e.g. `TOP_VOLANTIS` on Drogon. So it is a real issue.
The differences are in numerical stability for the optimization, and consequently large timing differences.

Remedy:
scale the data with the `StandardScaler` and then rescale either data or precision appropriately.
Likely:
- `fit_transform` data
- fit precision++
- transform / do the update
- `inverse_transform` the updated data

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.