Issue with sanitize_anndata() in plotting functions with subsets of anndata objects are passed.
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 2.6k
- Forks
- 779
- Avg merge
- 1d 4h
- Merged PRs (30d)
- 27
Description
I get quite a strange scanpy error, which appears a bit stochastic... This is has happened for the first time in version 1.1.
I am trying to get a scatter plot of a subsetted anndata object like this:
p4 = sc.pl.scatter(adata[adata.obs['n_counts']<10000 ,:], 'n_counts', 'n_genes', color='mt_frac')
When I do this the first time round, I get this error message about categorical variables from sanitize_anndata (none of which are actually used in the call).
AttributeError Traceback (most recent call last)
<ipython-input-66-fc1479c238f7> in <module>()
9 plt.show()
10
---> 11 p4 = sc.pl.scatter(adata[adata.obs['n_counts']<10000 ,:], 'n_counts', 'n_genes', color='mt_frac')
12 p5 = sc.pl.scatter(adata, 'n_counts', 'n_genes', color='mt_frac')
13
~/scanpy/scanpy/plotting/anndata.py in scatter(adata, x, y, color, use_raw, sort_order, alpha, basis, groups, components, projection, legend_loc, legend_fontsize, legend_fontweight, color_map, palette, right_margin, left_margin, size, title, show, save, ax)
162 show=show,
163 save=save,
--> 164 ax=ax)
165
166 elif x in adata.var_keys() and y in adata.var_keys() and color not in adata.obs_keys():
~/scanpy/scanpy/plotting/anndata.py in _scatter_obs(adata, x, y, color, use_raw, sort_order, alpha, basis, groups, components, projection, legend_loc, legend_fontsize, legend_fontweight, color_map, palette, right_margin, left_margin, size, title, show, save, ax)
281 ax=None):
282 """See docstring of scatter."""
--> 283 sanitize_anndata(adata)
284 if legend_loc not in VALID_LEGENDLOCS:
285 raise ValueError(
~/scanpy/scanpy/utils.py in sanitize_anndata(adata)
481 # backwards compat... remove this in the future
482 def sanitize_anndata(adata):
--> 483 adata._sanitize()
484
485
~/anndata/anndata/base.py in _sanitize(self)
1284 if len(c.categories) < len(c):
1285 df[key] = c
-> 1286 df[key].cat.categories = df[key].cat.categories.astype('U')
1287 logg.info(
1288 '... storing \'{}\' as categorical'
~/anaconda3/lib/python3.6/site-packages/pandas/core/generic.py in __getattr__(self, name)
3608 if (name in self._internal_names_set or name in self._metadata or
3609 name in self._accessors):
-> 3610 return object.__getattribute__(self, name)
3611 else:
3612 if name in self._info_axis:
~/anaconda3/lib/python3.6/site-packages/pandas/core/accessor.py in __get__(self, instance, owner)
52 # this ensures that Series.str.<method> is well defined
53 return self.accessor_cls
---> 54 return self.construct_accessor(instance)
55
56 def __set__(self, instance, value):
~/anaconda3/lib/python3.6/site-packages/pandas/core/categorical.py in _make_accessor(cls, data)
2209 def _make_accessor(cls, data):
2210 if not is_categorical_dtype(data.dtype):
-> 2211 raise AttributeError("Can only use .cat accessor with a "
2212 "'category' dtype")
2213 return CategoricalAccessor(data.values, data.index,
AttributeError: Can only use .cat accessor with a 'category' dtype
Then, I comment out the respective line of code, run the whole thing again, and it works. And when I uncomment the line it works fine again.
When I comment the line for the first time, I get a couple of lines displayed in the output saying:
... 'donor' was turned into a categorical variable
... 'gene_symbols' was turned into a categorical variable
or something like that...
My theory is that sanitize_anndata() detects that these variables should be categorical variables and tries to convert them into categoricals. As this sc.pl.scatter call is the first time sanitize_anndata() is called after the variables are read in, this is the first time this conversion would take place. However, I am calling the sc.pl.scatter() on a subsetted anndata object, so it somehow cannot do the conversion. Once I call sc.pl.scatter on a non-subsetted anndata object once, the conversion can take place and I can subsequently call sc.pl.scatter also on a subsetted anndata object.
If this is true, I can see why this is happening. However I feel this behaviour will be quite puzzling to a typical user. Maybe sanitize_anndata() should be called before plotting (probably hard to implement), or the plotting functions should have a parameter to plot only a subset of the data. That way sanitize_anndata can be called on the whole anndata object every time as there is no longer a reason to pass a view of the object. You could then test if a view is being passed to sanitize anndata, and then say "please don't pass subsetted anndata objects to plotting functions" or something like that.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the subsetted scatter call in scanpy/plotting/anndata.py and trace sanitize_anndata in scanpy/utils.py into anndata/base.py. Compare the first call on a view with a call on the full AnnData object, focusing on categorical conversion. Done means plotting a subset no longer raises the reported pandas AttributeError and the conversion behavior is covered or clearly documented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data-visualization
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100