numpy / numpy/numpy

vectorize does not play well with pydoc and Sphinx

Open
#13,826 0 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
32.8k
Forks
12.8k
Avg merge
1d 7h
Merged PRs (30d)
197

Description

The numpy.vectorize documentation states "The docstring is taken from the input function to vectorize unless it is specified". However, vectorize is implemented in numpy as a class (with a __call__ method to make the result function-like), and sets __doc__ as an instance attribute. The same probably applies to other ufunc-based features in numpy. Unfortunately, none of Python's documentation systems have support for instance docstrings, or at least not pydoc (the built-in help or ?/?? in IPython) and Sphinx. As a results, help shows the class docstring, which is the documentation of numpy.vectorize, not the instance docstring of the vectorized function. See http://numpy-discussion.10968.n7.nabble.com/numpy-vectorize-docstrings-not-shown-by-help-command-tt38551.html and https://stackoverflow.com/questions/29577849/docstring-mismatch-using-numpys-vectorize.
Thus, while numpy's documentation is correct to the letter (the __doc__ attribute is preserved), relying on non-existent support for instance docstrings makes the feature rather pointless.

I do think that pydoc/Sphinx should have support for instance docstrings (although I wasn't able to find any open issue indicating that such a feature is currently planned). However, even if this support were to be added, numpy's implementation of vectorize and other ufunc-based features is still questionable: Sphinx does categorization when generating the documentation, see https://www.sphinx-doc.org/en/master/usage/extensions/autodoc.html, distinguishing between classes, functions, exceptions, data, etc. The problem with numpy.vectorize is that it moves the decorated function from the "function" category to the "data" category, which (depending on the details of how Sphinx is used) likely causes the vectorize-decorated functions to not be included in the documentation at all.

Thus, I would strongly recommend that numpy should not use classes as function decorators: all this breakage of the documentation systems can be avoided by implementing the decorator as a function with the appropriate closures instead.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the numpy.vectorize entry point and compare its class-based decorator behavior with how pydoc and Sphinx categorize decorated functions. Determine whether the proposed implementation change can preserve the input function's documentation while keeping vectorized functions discoverable as functions; done means the documented mismatch is resolved for vectorize.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
documentation
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.