fslaborg / fslaborg/FSharp.Stats

Ways to deal with nan in matrix operations

Open
#113 1 comment 0 reactions 0 assignees View on GitHub
help wanted up-for-grabs
Dominant language
F#
Stars
227
Forks
58
Avg merge
55m
Merged PRs (30d)
1

Description

**Is your feature request related to a problem? Please describe.**
In R, when you want to calculate a variance-covariance matrix, you can pass in options via a "use" for what to do in the case of missing values: https://www.rdocumentation.org/packages/stats/versions/3.6.2/topics/cor . For instance, use="pairwise.complete".

It would be nice to have this option in FSharp.Stats and have it work for float nan and float Option types. Though I'm not sure how best to mix new parameters into the Matrix operations api. Something along the lines of the Plotly.NET options might work, Matrix.columnSampleCovarianceMatrixOf Complete A, Matrix.columnSampleCovarianceMatrixOf PairwiseComplete A, etc.

**Describe the solution you'd like**

For example, in R I can do

```R
> A <- matrix(c(1,2,1.1,2.9,NA,1.5),ncol = 2, byrow = TRUE)
> A
[,1] [,2]
[1,] 1.0 2.0
[2,] 1.1 2.9
[3,] NA 1.5
> cov(A)
[,1] [,2]
[1,] NA NA
[2,] NA 0.5033333
> cov(A,use="pairwise.complete")
[,1] [,2]
[1,] 0.005 0.0450000
[2,] 0.045 0.5033333
```

In FSharp.Stats right now, I have to do the following, and it is not straightforward to do it programmatically with functions like Matrix.mapiRow.
```fsharp
> let A = matrix [[1.0;2.0];[1.1;2.9];[nan; 1.5]];;
val A : Matrix = matrix [[1.0; 2.0]
[1.1; 2.9]
[nan; 1.5]]

> let completeCov = Matrix.columnSampleCovarianceMatrixOf A;;
val completeCov : Matrix = matrix [[nan; nan]
[nan; 0.5033333333]]

> completeCov ;;
val it : Matrix = matrix [[nan; nan]
[nan; 0.5033333333]]

> // To get pairwise complete
- ;;
> let pairwiseCompleteA = Matrix.removeRowAt 2 A;;
val pairwiseCompleteA : Matrix = matrix [[1.0; 2.0]
[1.1; 2.9]]

> let pairwiseCov = Matrix.columnSampleCovarianceMatrixOf pairwiseCompleteA;;
val pairwiseCov : Matrix = matrix [[0.005; 0.045]
[0.045; 0.405]]

> pairwiseCov;;
val it : Matrix = matrix [[0.005; 0.045]
[0.045; 0.405]]

> pairwiseCov.Item (1,1) <- completeCov.Item (1,1);;
val it : unit = ()

> pairwiseCov;;
val it : Matrix = matrix [[0.005; 0.045]
[0.045; 0.5033333333]]
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.