lemur function force conversion to dense matrix
- Dominant language
- R
- Stars
- 102
- Forks
- 11
- PR merge metrics
- No merged PRs in 30d
Description
Hello!
Thanks for the interesting package!
I'm testing it on one of our datasets, which contains about 1M cells and 1.5k individuals. When calling the `lemur` function, I noticed that the tool converts the sparse count matrix into a dense array. This occupies a huge amount of memory and thus limits the possibility of applying the method to large-scale data.
Briefly, I have a SingleCellExperiment object that I generated from a dgCMatrix, and data tables like this.
```R
sce <- SingleCellExperiment(
assays = list(logcounts = sparse_matrix),
colData = cell_info,
rowData = gene_info
)
```
Then I'm trying to run lemur as suggested in the tutorial
```R
fit <- lemur(sce, design = ~ phenotype + Exp,
n_embedding = 20, test_fraction = 0.5)
```
And I get this message:
```
Warning: sparse->dense coercion: allocating vector of size 172.0 GiB
```
The memory occupancy then quickly goes up to about 250 GB.
Am I doing something wrong? Is there a way to make the method work with a sparse matrix (the standard dgCMatrix used by SingleCellExperiment)?
Thanks!
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.