python / python/cpython

object.c inconsistently truncates tp_name in exception error messages, hampering debugging

Open
#101,964 4 comments 0 reactions 1 assignee View on GitHub

@gpshead is already working on this.

Since Feb 16, 2023.

interpreter-core type-feature
Dominant language
Python
Stars
77.2k
Forks
36k
PR merge metrics
PR metrics pending

Description

Objects/object.c places tp_name into TypeError and AttributeError exception messages via an inconsistent mess of %.50s, %.100s, and %.200s PyErr_FromFormat format string specifiers.

We butt heads with this at work routinely. Especially on the mere 50 character limit cases. When wrapping C++ classes within a deep directory tree or namespace the .tp_name of the class your pybind11 or otherwise generated extension module code will use can easily be longer than that.

We're going ahead with a local patch to make these consistent with a limit of 200 internally for now, which I'd like to apply here.

But the question arises: What practical reason do we limit the rendered size of tp_name for at all?

Truncating is always the wrong thing to do for information within an error message, the beginning and end of a name string are usually the most unique deterministic parts useful as a debugging aid. If we are to truncate at all rather than just use an unbounded %s format for tp_name we should elide characters from the middle with a ... placeholder. If we don't have a format specifier that can do that for PyUnicodeObject, one would need to be added.

Is there ever a situation where a C PyTypeObject's tp_name can be assigned to an arbitrary length string at runtime?

(There may be places outside of object.c doing this with tp_name as well, I consider finding those and being consistent in approach there part of this issue)


This theme has come up in different areas in the past, but those were for general str/repr of things which are larger cans of worms. Not the tp_name field. For example #57558 and other things generally mentioned via #51579 which added our length specifiers to PyUnicode formatting.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.