scverse / scverse/scanpy

Dask support

Open
#2,578 3 comments 0 reactions 1 assignee View on GitHub

@flying-sheep is already working on this.

Since Aug 4, 2023.

Area - Out of core 💾 Area – Performance 🐌 Meta
Dominant language
Python
Stars
2.6k
Forks
779
Avg merge
1d 4h
Merged PRs (30d)
27

Description

What kind of feature would you like to request?

Additional function parameters / changed functionality / changed defaults?

Please describe your wishes

Meta issue tracking scanpy functions without dask support

Related #921

First we should test things more generically:

  • add global array_type fixture to use in all tests for features below
    • #2595

@ivirshup came up with places where we should prioritize dask support:

  • normalize_total
    • seems to work: scverse/scanpy#2466
    • has tests: scverse/scanpy#1663
  • log1p, normalize_per_cell, filter_cells/*_genes:
    • accept Dask arrays: scverse/scanpy#283
    • Test that they are kept as Dask arrays throughout: scverse/scanpy#2814
  • sc.pp.pca: scverse/scanpy#2563 scverse/scanpy#3263
  • sc.pp.calculate_qc_metrics #3307
  • sc.pp.highly_variable_genes:
    • scverse/scanpy#2777
    • scverse/scanpy#2807
    • scverse/scanpy#2808
    • experimental “pearson_residuals” flavor
  • sc.tl.rank_genes_groups
    • depends on #2621
  • sc.pp.subsample/sc.pp.sample: so simple it has always been supported
  • Aggregate: https://flox.readthedocs.io/en/latest/ #3659
    • CSC: #3872

Later:

  • scale scverse/scanpy#2942
  • nearest-neighbors: https://github.com/dask/dask-ml/issues/982 may need to implement ourselves...
  • umap: implement ourselves?
  • metrics: depednent on squidpy decisions but should be dask-ifiable
  • scrublet?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.