scverse / scverse/scanpy

Reading 10x scRNA-seq with read_10x_mtx()

Open
#1,916 14 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Needs info❔
Dominant language
Python
Stars
2.6k
Forks
779
Avg merge
1d 4h
Merged PRs (30d)
27

Description

I get an error when I use sc.read_10x_mtx() to read scRNA-seq files (matrix.mtx.gz, barcodes.tsv.gz, and features.tsv.gz) generated by "Illumina HiSeq 4000". The function works fine when I read files generated by other 10x platforms such as "Illumina NovaSeq 6000".

Example:

sc.read_10x_mtx("GSE145328_RAW")

Error:

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
~/anaconda3/lib/python3.8/site-packages/pandas/core/indexes/base.py in get_loc(self, key, method, tolerance)
   3079             try:
-> 3080                 return self._engine.get_loc(casted_key)
   3081             except KeyError as err:

pandas/_libs/index.pyx in pandas._libs.index.IndexEngine.get_loc()

pandas/_libs/index.pyx in pandas._libs.index.IndexEngine.get_loc()

pandas/_libs/hashtable_class_helper.pxi in pandas._libs.hashtable.Int64HashTable.get_item()

pandas/_libs/hashtable_class_helper.pxi in pandas._libs.hashtable.Int64HashTable.get_item()

KeyError: 2

The above exception was the direct cause of the following exception:

KeyError                                  Traceback (most recent call last)
<ipython-input-20-26443e0aed95> in <module>
----> 1 rnaseq1 = sc.read_10x_mtx("GSE145328_RAW")

~/anaconda3/lib/python3.8/site-packages/scanpy/readwrite.py in read_10x_mtx(path, var_names, make_unique, cache, cache_compression, gex_only, prefix)
    479     genefile_exists = (path / f'{prefix}genes.tsv').is_file()
    480     read = _read_legacy_10x_mtx if genefile_exists else _read_v3_10x_mtx
--> 481     adata = read(
    482         str(path),
    483         var_names=var_names,

~/anaconda3/lib/python3.8/site-packages/scanpy/readwrite.py in _read_v3_10x_mtx(path, var_names, make_unique, cache, cache_compression, prefix)
    560     else:
    561         raise ValueError("`var_names` needs to be 'gene_symbols' or 'gene_ids'")
--> 562     adata.var['feature_types'] = genes[2].values
    563     adata.obs_names = pd.read_csv(path / f'{prefix}barcodes.tsv.gz', header=None)[
    564         0

~/anaconda3/lib/python3.8/site-packages/pandas/core/frame.py in __getitem__(self, key)
   3022             if self.columns.nlevels > 1:
   3023                 return self._getitem_multilevel(key)
-> 3024             indexer = self.columns.get_loc(key)
   3025             if is_integer(indexer):
   3026                 indexer = [indexer]

~/anaconda3/lib/python3.8/site-packages/pandas/core/indexes/base.py in get_loc(self, key, method, tolerance)
   3080                 return self._engine.get_loc(casted_key)
   3081             except KeyError as err:
-> 3082                 raise KeyError(key) from err
   3083 
   3084         if tolerance is not None:

KeyError: 2
Versions

scanpy==1.8.0 anndata==0.7.6 umap==0.5.1 numpy==1.19.2 scipy==1.6.3 pandas==1.2.4 scikit-learn==0.24.2 statsmodels==0.12.2 python-igraph==0.9.1 pynndescent==0.5.2

[Paste the output of scanpy.logging.print_versions() leaving a blank line after the details tag]

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in scanpy/readwrite.py at read_10x_mtx() and _read_v3_10x_mtx(), then compare the HiSeq 4000 features.tsv.gz structure with the files accepted from other 10x platforms. Reproduce the reported KeyError with GSE145328_RAW and verify that the reader handles the input while preserving the expected feature metadata.

Written by the indexing model from the issue text.

Assessment

Tech stack
pandas, python
Domain
data
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.