sphinx-doc / sphinx-doc/sphinx

sphinx.ext.autodoc treats some class attributes as class methods depending on assignment

Open
#10,293 0 comments 4 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

extensions:autodoc type:bug
Dominant language
Python
Stars
8k
Forks
2.6k
PR merge metrics
No merged PRs in 30d

Description

Describe the bug

This may or may not be a bug, but there are situations in which a class attribute becomes hidden from sphinx if it's rvalue is a class. In our documentation for pyhf, we have some classes that need to bookkeep the array type, e.g. something like this

class numpy_backend:
  #: this is the numpy array type
  array_type = np.ndarray

Our api.rst looks like this

Backends
--------

The computational backends that :code:`pyhf` provides interfacing for the vector-based calculations.

.. currentmodule:: pyhf.tensor

.. autosummary::
   :toctree: _generated/
   :nosignatures:

   numpy_backend.numpy_backend

where numpy_backend.numpy_backend is a class. If we do not specify any autosummary templates and use the default, the output looks like this:

Screen Shot 2022-03-24 at 4 47 49 PM

Somewhat confusingly, array_type does not show up at all, not even in the "methods". It's just gone.

If we instead try to provide an autosummary template to really make sure the attributes get in there, e.g. autosummary/class.rst looks like

{{ name | escape | underline}}

.. currentmodule:: {{ module }}

.. autoclass:: {{ objname }}
   :show-inheritance:

   {% block attributes %}
   {% if attributes %}
   .. rubric:: {{ _('Attributes') }}

   {% for item in attributes %}
   .. autoattribute:: {{ name }}.{{ item }}
   {%- endfor %}
   {% endif %}
   {% endblock %}

   {% block methods %}

   {% if methods %}
   .. rubric:: {{ _('Methods') }}

   {% for item in members %}
   {% if item not in attributes and not item.startswith('__') %}
   .. automethod:: {{ name }}.{{ item }}
   {% endif %}
   {%- endfor %}

   {% endif %}
   {% endblock %}

We end up with a warning like this in Sphinx generation:

/Users/kratsg/pyhf/src/pyhf/tensor/numpy_backend.py:docstring of pyhf.tensor.numpy_backend.numpy_backend.shape:1: WARNING: duplicate object description of pyhf.tensor.numpy_backend.numpy_backend.shape, other instance in _generated/pyhf.tensor.numpy_backend.numpy_backend, use :noindex: for one of them

where it seems like Sphinx's automethod has picked up this class attribute and started documenting the class itself

Screen Shot 2022-03-24 at 4 51 25 PM

which is somewhat surprising, but ok, this seems to indicate that Sphinx is completely not seeing this class attribute as an attribute at all. So instead, we modify the class.rst above to add in blocks to autoattribute this attribute if it exists

   {% if 'array_type' in members %}
   .. autoattribute:: {{ name }}.array_type
   {% endif %}

and to modify the members to skip it as well

   {% for item in members %}
   {% if item not in attributes and not item.startswith('__') and item not in ['array_type'] %}
   .. automethod:: {{ name }}.{{ item }}
   {% endif %}
   {%- endfor %}

but this seems like a very explicit hack to get around this. Is there particularly some way of telling Sphinx that this array_type is indeed an attribute and not a class it needs to document?

One other option is to rewrite the class to use @property instead to at least force sphinx to parse it correctly

class numpy_backend:
  @property
  def array_type(self):
    """this is the numpy array type"""
    return np.ndarray

but this means at least in our codebase, we'd need to do numpy_backend.array_type.fget(None) to at least get the value out which is a bit long-winded and slightly ugly.


Even more confusingly, if we look at the other backends, they don't have this problem that numpy's does!

class tensorflow_backend:
  array_type = tensorflow.Tensor

Sphinx recognizes this just fine as an attribute. Is there some special going on with numpy? This means we really have to add some extra-extra checks here to make sure we don't double-count the attributes for all but numpy!

   {% if name == 'numpy_backend' %}
     {% if 'array_type' in members %}
   .. autoattribute:: {{ name }}.array_type
     {% endif %}
     {% if 'array_subtype' in members %}
   .. autoattribute:: {{ name }}.array_subtype
     {% endif %}
   {% endif %}

This gets really winded quickly. Not sure what's special about numpy classes versus other classes. Are we doing something really wrong with autoclass or is there something special about np.ndarray and np.number that Sphinx is picking up and treating as a method on the class, and not the value of an attribute on the class?

How to Reproduce
$ git clone -b feat/jsonschema_tensorValidation_sphinxDocIssue https://github.com/scikit-hep/pyhf
$ cd pyhf
$ pip install .[docs]
$ cd docs
$ make html
$ # open _build/html/index

play around with the docs/_template/autosummary/class.rst with the above and see the variations.

Expected behavior

We expect array_type (a class attribute) to be treated as a class attribute in Sphinx to get output like this

Screen Shot 2022-03-24 at 5 04 55 PM

Your project

https://github.com/scikit-hep/pyhf/tree/feat/jsonschema_tensorValidation_sphinxDocIssue

Screenshots

No response

OS

Mac 10.14.6

Python version

3.8.6

Sphinx version

4.4.0

Sphinx extensions

['sphinx.ext.autodoc', 'sphinx.ext.autosummary', 'sphinx.ext.coverage', 'sphinx.ext.mathjax', 'sphinx.ext.ifconfig', 'sphinx.ext.viewcode', 'sphinx.ext.githubpages', 'sphinx.ext.intersphinx', 'sphinxcontrib.bibtex', 'sphinx.ext.napoleon', 'sphinx_click.ext', 'nbsphinx', 'sphinx_issues', 'sphinx_copybutton', 'xref']

Extra tools

No response

Additional context

No response

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Run the pyhf reproduction with Sphinx 4.4.0 using cd docs && make html, then compare the default and custom autosummary/class.rst outputs. Trace how sphinx.ext.autodoc and sphinx.ext.autosummary classify array_type when its value is np.ndarray versus tensorflow.Tensor. Done means class-valued attributes are consistently documented as attributes without duplicate method descriptions.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
documentation
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.