scverse / scverse/scanpy

Issue with sanitize_anndata() in plotting functions with subsets of anndata objects are passed.

Open
#166 6 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Area - Plotting 🌺
Dominant language
Python
Stars
2.6k
Forks
779
Avg merge
1d 4h
Merged PRs (30d)
27

Description

I get quite a strange scanpy error, which appears a bit stochastic... This is has happened for the first time in version 1.1.

I am trying to get a scatter plot of a subsetted anndata object like this:
p4 = sc.pl.scatter(adata[adata.obs['n_counts']<10000 ,:], 'n_counts', 'n_genes', color='mt_frac')

When I do this the first time round, I get this error message about categorical variables from sanitize_anndata (none of which are actually used in the call).

AttributeError                            Traceback (most recent call last)
<ipython-input-66-fc1479c238f7> in <module>()
      9 plt.show()
     10 
---> 11 p4 = sc.pl.scatter(adata[adata.obs['n_counts']<10000 ,:], 'n_counts', 'n_genes', color='mt_frac')
     12 p5 = sc.pl.scatter(adata, 'n_counts', 'n_genes', color='mt_frac')
     13 

~/scanpy/scanpy/plotting/anndata.py in scatter(adata, x, y, color, use_raw, sort_order, alpha, basis, groups, components, projection, legend_loc, legend_fontsize, legend_fontweight, color_map, palette, right_margin, left_margin, size, title, show, save, ax)
    162                 show=show,
    163                 save=save,
--> 164                 ax=ax)
    165 
    166         elif x in adata.var_keys() and y in adata.var_keys() and color not in adata.obs_keys():

~/scanpy/scanpy/plotting/anndata.py in _scatter_obs(adata, x, y, color, use_raw, sort_order, alpha, basis, groups, components, projection, legend_loc, legend_fontsize, legend_fontweight, color_map, palette, right_margin, left_margin, size, title, show, save, ax)
    281         ax=None):
    282     """See docstring of scatter."""
--> 283     sanitize_anndata(adata)
    284     if legend_loc not in VALID_LEGENDLOCS:
    285         raise ValueError(

~/scanpy/scanpy/utils.py in sanitize_anndata(adata)
    481 # backwards compat... remove this in the future
    482 def sanitize_anndata(adata):
--> 483     adata._sanitize()
    484 
    485 

~/anndata/anndata/base.py in _sanitize(self)
   1284                     if len(c.categories) < len(c):
   1285                         df[key] = c
-> 1286                         df[key].cat.categories = df[key].cat.categories.astype('U')
   1287                         logg.info(
   1288                             '... storing \'{}\' as categorical'

~/anaconda3/lib/python3.6/site-packages/pandas/core/generic.py in __getattr__(self, name)
   3608         if (name in self._internal_names_set or name in self._metadata or
   3609                 name in self._accessors):
-> 3610             return object.__getattribute__(self, name)
   3611         else:
   3612             if name in self._info_axis:

~/anaconda3/lib/python3.6/site-packages/pandas/core/accessor.py in __get__(self, instance, owner)
     52             # this ensures that Series.str.<method> is well defined
     53             return self.accessor_cls
---> 54         return self.construct_accessor(instance)
     55 
     56     def __set__(self, instance, value):

~/anaconda3/lib/python3.6/site-packages/pandas/core/categorical.py in _make_accessor(cls, data)
   2209     def _make_accessor(cls, data):
   2210         if not is_categorical_dtype(data.dtype):
-> 2211             raise AttributeError("Can only use .cat accessor with a "
   2212                                  "'category' dtype")
   2213         return CategoricalAccessor(data.values, data.index,

AttributeError: Can only use .cat accessor with a 'category' dtype

Then, I comment out the respective line of code, run the whole thing again, and it works. And when I uncomment the line it works fine again.

When I comment the line for the first time, I get a couple of lines displayed in the output saying:

... 'donor' was turned into a categorical variable
... 'gene_symbols' was turned into a categorical variable

or something like that...

My theory is that sanitize_anndata() detects that these variables should be categorical variables and tries to convert them into categoricals. As this sc.pl.scatter call is the first time sanitize_anndata() is called after the variables are read in, this is the first time this conversion would take place. However, I am calling the sc.pl.scatter() on a subsetted anndata object, so it somehow cannot do the conversion. Once I call sc.pl.scatter on a non-subsetted anndata object once, the conversion can take place and I can subsequently call sc.pl.scatter also on a subsetted anndata object.

If this is true, I can see why this is happening. However I feel this behaviour will be quite puzzling to a typical user. Maybe sanitize_anndata() should be called before plotting (probably hard to implement), or the plotting functions should have a parameter to plot only a subset of the data. That way sanitize_anndata can be called on the whole anndata object every time as there is no longer a reason to pass a view of the object. You could then test if a view is being passed to sanitize anndata, and then say "please don't pass subsetted anndata objects to plotting functions" or something like that.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the subsetted scatter call in scanpy/plotting/anndata.py and trace sanitize_anndata in scanpy/utils.py into anndata/base.py. Compare the first call on a view with a call on the full AnnData object, focusing on categorical conversion. Done means plotting a subset no longer raises the reported pandas AttributeError and the conversion behavior is covered or clearly documented.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data-visualization
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.