"batch_size" parameter doesn't work for "sc.external.pp.scanorama_integrate"
Open
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 2.6k
- Forks
- 779
- Avg merge
- 1d 4h
- Merged PRs (30d)
- 27
Description
According to scanorama MemoryError, I think batch_size parameter doesn't work for sc.external.pp.scanorama_integrate
Minimal code sample (that we can copy&paste without having any data)
sc.external.pp.scanorama_integrate(adata, key="datasetID", batch_size=500)
WARNING: Out of memory, consider turning on batched computation with batch_size parameter.
Traceback (most recent call last):
File "/lustre1/shiq//02_igt/py/1_igt_scanorama.py", line 26, in <module>
sc.external.pp.scanorama_integrate(adata, key="datasetID", batch_size=500)
File "/lustre1/shiq/app/miniconda3/envs/scanpy/lib/python3.10/site-packages/scanpy/external/pp/_scanorama_integrate.py", line 121, in scanorama_integrate
integrated = scanorama.assemble(
File "/lustre1/shiq/app/miniconda3/envs/scanpy/lib/python3.10/site-packages/scanorama/scanorama.py", line 933, in assemble
bias = transform(curr_ds, curr_ref, ds_ind, ref_ind, sigma=sigma,
File "/lustre1/shiq/app/miniconda3/envs/scanpy/lib/python3.10/site-packages/scanorama/scanorama.py", line 762, in transform
avg_bias = batch_bias(curr_ds, match_ds, bias, sigma=sigma,
File "/lustre1/shiq/app/miniconda3/envs/scanpy/lib/python3.10/site-packages/scanorama/scanorama.py", line 723, in batch_bias
weights = rbf_kernel(curr_ds, match_ds, gamma=0.5*sigma)
File "/lustre1/shiq/app/miniconda3/envs/scanpy/lib/python3.10/site-packages/sklearn/metrics/pairwise.py", line 1294, in rbf_kernel
X, Y = check_pairwise_arrays(X, Y)
File "/lustre1/shiq/app/miniconda3/envs/scanpy/lib/python3.10/site-packages/sklearn/metrics/pairwise.py", line 155, in check_pairwise_arrays
X = check_array(
File "/lustre1/shiq/app/miniconda3/envs/scanpy/lib/python3.10/site-packages/sklearn/utils/validation.py", line 727, in check_array
warnings.warn(
FutureWarning: np.matrix usage is deprecated in 1.0 and will raise a TypeError in 1.2. Please convert to a numpy array with np.asarray. For more information see: https://numpy.org/doc/stable/reference/generated/numpy.matrix.html
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Read scanpy/external/pp/_scanorama_integrate.py, especially the scanorama_integrate entry point and its scanorama.assemble call, then trace batch_size into the dependency’s assemble/transform path. Reproduce the provided call and traceback; done means the parameter reaches the intended batched computation without the reported out-of-memory path.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- numpy, python, scikit-learn
- Domain
- data
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100