fslaborg / fslaborg/FSharp.Stats
Ways to deal with nan in matrix operations
- Dominant language
- F#
- Stars
- 227
- Forks
- 58
- Avg merge
- 55m
- Merged PRs (30d)
- 1
Description
**Is your feature request related to a problem? Please describe.**
In R, when you want to calculate a variance-covariance matrix, you can pass in options via a "use" for what to do in the case of missing values: https://www.rdocumentation.org/packages/stats/versions/3.6.2/topics/cor . For instance, use="pairwise.complete".
It would be nice to have this option in FSharp.Stats and have it work for float nan and float Option types. Though I'm not sure how best to mix new parameters into the Matrix operations api. Something along the lines of the Plotly.NET options might work, Matrix.columnSampleCovarianceMatrixOf Complete A, Matrix.columnSampleCovarianceMatrixOf PairwiseComplete A, etc.
**Describe the solution you'd like**
For example, in R I can do
```R
> A <- matrix(c(1,2,1.1,2.9,NA,1.5),ncol = 2, byrow = TRUE)
> A
[,1] [,2]
[1,] 1.0 2.0
[2,] 1.1 2.9
[3,] NA 1.5
> cov(A)
[,1] [,2]
[1,] NA NA
[2,] NA 0.5033333
> cov(A,use="pairwise.complete")
[,1] [,2]
[1,] 0.005 0.0450000
[2,] 0.045 0.5033333
```
In FSharp.Stats right now, I have to do the following, and it is not straightforward to do it programmatically with functions like Matrix.mapiRow.
```fsharp
> let A = matrix [[1.0;2.0];[1.1;2.9];[nan; 1.5]];;
val A : Matrix = matrix [[1.0; 2.0]
[1.1; 2.9]
[nan; 1.5]]
> let completeCov = Matrix.columnSampleCovarianceMatrixOf A;;
val completeCov : Matrix = matrix [[nan; nan]
[nan; 0.5033333333]]
> completeCov ;;
val it : Matrix = matrix [[nan; nan]
[nan; 0.5033333333]]
> // To get pairwise complete
- ;;
> let pairwiseCompleteA = Matrix.removeRowAt 2 A;;
val pairwiseCompleteA : Matrix = matrix [[1.0; 2.0]
[1.1; 2.9]]
> let pairwiseCov = Matrix.columnSampleCovarianceMatrixOf pairwiseCompleteA;;
val pairwiseCov : Matrix = matrix [[0.005; 0.045]
[0.045; 0.405]]
> pairwiseCov;;
val it : Matrix = matrix [[0.005; 0.045]
[0.045; 0.405]]
> pairwiseCov.Item (1,1) <- completeCov.Item (1,1);;
val it : unit = ()
> pairwiseCov;;
val it : Matrix = matrix [[0.005; 0.045]
[0.045; 0.5033333333]]
```
Contributor guide
Assessment
This issue has not been assessed yet.