Docs: "Extending Python with C or C++" explanation of PY_SSIZE_T_CLEAN explains nothing

Start with the “Extending Python with C or C++” chapter, especially “A Simple Example” and “Extracting Parameters in Extension Functions,” then compare them with the linked Python/C API reference section on strings and buffers. Improve the PY_SSIZE_T_CLEAN explanation and links so a newcomer understands its effect and when to define it before Python.h.

Documentation

The manual chapter "Extending Python with C or C++" opens with "A Simple Example", first presenting these two lines of source:

#define PY_SSIZE_T_CLEAN
#include <Python.h>

This is shortly followed by a note:

Note Since Python may define some pre-processor definitions which affect the standard headers on some systems, you must include Python.h before any standard headers are included.

It is recommended to always define PY_SSIZE_T_CLEAN before including Python.h. See Extracting Parameters in Extension Functions for a description of this macro.

The problem

Is, in a nutshell, with the latter half of that note.

The section referenced, "Extracting Parameters in Extension Functions" contains no real explanation of PY_SSIZE_T_CLEAN at all; the only difference is that the #define in its example code is now followed by a comment:

#define PY_SSIZE_T_CLEAN  /* Make "s#" use Py_ssize_t rather than int. */
#include <Python.h>

I would submit that this "explanation" explains nothing, unless the user already knows about the PyArg_Parse*() family of functions and understands their byzantine format syntax. To anyone else, "s#" (a PyArg_Parse() format string token — concepts which haven't even been introduced by that point) has no obvious meaning, to say nothing of the type it "uses". Even an experienced C/C++ programmer, unfamiliar with Python's C API, can't really be expected to get anything useful out of that comment.

The "Extracting Parameters..." section does also direct the reader to the Python/C API reference. (Just generally, for information about argument parsing. Not in any way that explicitly or obviously relates to PY_SSIZE_T_CLEAN.)

If the user chooses to explore, there they will find an actual explanation of PY_SSIZE_T_CLEAN (though it still requires reading a fair amount of the surrounding context to understand):

(from https://docs.python.org/3/c-api/arg.html#strings-and-buffers):

Note For all # variants of formats (s#, y#, etc.), the macro PY_SSIZE_T_CLEAN must be defined before including Python.h. On Python 3.9 and older, the type of the length argument is Py_ssize_t if the PY_SSIZE_T_CLEAN macro is defined, or int otherwise.

Proposed fix

It might be better to just point them at the relevant section of the reference manual right from the start, rather than giving them the runaround through "Extracting Parameters..." and hoping they're curious enough to follow a second link.

Or, if dropping them in the deep end of PyArg_Parse() right away is too daunting for new API users (which it absolutely, totally is), then perhaps "Extracting Parameters..." should include some sort of real (if handwavey) explanation of PY_SSIZE_T_CLEAN, along with a link to the reference manual. Something like:

Note The macro PY_SSIZE_T_CLEAN must be defined before including Python.h, to ensure backwards compatibility in Python's argument parsing functions.

In Python 3.9 and older, when a variable-size buffer argument is parsed, its size will be returned as a type Py_ssize_t value if PY_SSIZE_T_CLEAN is set, or as a type int if it is unset. Python 3.10 and later will always use Py_ssize_t for buffer size values.

See Parsing arguments and building values in the Python/C API Reference Manual for more information about the PyArg_Parse() functions and how PY_SSIZE_T_CLEAN interacts with format strings containing the sized tokens s#, y#, etc.

In new code, store and pass argument buffer sizes exclusively as type Py_ssize_t, and always #define PY_SSIZE_T_CLEAN before including Python.h to ensure compatibility with older Python releases.