equinor / equinor/graphite-maps
Scale data before precision estimation
- Dominant language
- Python
- Stars
- 8
- Forks
- 3
- PR merge metrics
- No merged PRs in 30d
Description
There can be a large difference between
```python
Prec_u_sub = fit_precision_cholesky(X, graph_u_sub, verbose_level=5)
```
and
```python
from sklearn.preprocessing import StandardScaler
scaler = StandardScaler()
# Fit the scaler to the data and transform it
X_scaled = scaler.fit_transform(X)
X_scaled.shape
Prec_u_sub = fit_precision_cholesky(X_scaled, graph_u_sub, verbose_level=5)
```
In particular, this is experienced on e.g. `TOP_VOLANTIS` on Drogon. So it is a real issue.
The differences are in numerical stability for the optimization, and consequently large timing differences.
Remedy:
scale the data with the `StandardScaler` and then rescale either data or precision appropriately.
Likely:
- `fit_transform` data
- fit precision++
- transform / do the update
- `inverse_transform` the updated data
Contributor guide
Assessment
This issue has not been assessed yet.