LOAD_ATTR specialization for ModuleType subclasses bypasses data descriptors on the subclass (3.14 regression)
Dieses Issue hat noch niemand übernommen.
Bewertung
- Schwierigkeit
- 4/5
- Geschätzter Aufwand
- 3-5 Tage
- Anfängerfreundlichkeit
- 35/100
Rechercherichtung
Beginne in Python/specialize.c bei _Py_Specialize_LoadAttr und specialize_module_load_attr_lock_held, und prüfe anschließend den im Bericht beschriebenen LOAD_ATTR_MODULE-Guard. Führe den bereitgestellten Reproducer für die ModuleType-Unterklasse über die betroffenen Versionen hinweg aus und sieh dir bestehende Spezialisierungstests an. Erledigt ist die Aufgabe, wenn wiederholte Attributzugriffe den Daten-Deskriptor der Unterklasse konsistent berücksichtigen und eine Regressionstestabdeckung vorhanden ist.
Vom Indexierungsmodell aus dem Issue-Text verfasst.
Beschreibung
Bug report
Bug description:
Since 3.14, attribute loads on an instance of a types.ModuleType
subclass are specialized to LOAD_ATTR_MODULE, which reads the module
dictionary directly. The specialization does not check whether the
subclass defines a data descriptor for the name, so after the first
access a data descriptor on the subclass is silently bypassed and the
dictionary value is returned instead. Before 3.14 the guard was
PyModule_CheckExact, so subclasses never took this path and the
descriptor was honoured on every access.
The following runs the reproducer on 3.13, 3.14, 3.14t, 3.15 and
3.15t using uv:
for v in 3.13 3.14 3.14t 3.15 3.15t; do uv run --no-project --python $v python - <<'EOF'
import sys
import sysconfig
import types
class Desc:
def __get__(self, instance, owner=None):
return "from descriptor"
def __set__(self, instance, value):
instance.__dict__["x"] = value
class Module(types.ModuleType):
pass
Module.x = Desc()
m = Module("m")
m.__dict__["x"] = "from dict"
build = "free-threaded" if sysconfig.get_config_var("Py_GIL_DISABLED") else "default"
print(sys.version.split()[0], build, [m.x for _ in range(4)])
EOF
done
Output:
3.13.15 default ['from descriptor', 'from descriptor', 'from descriptor', 'from descriptor']
3.14.7 default ['from descriptor', 'from dict', 'from dict', 'from dict']
3.14.7 free-threaded ['from descriptor', 'from dict', 'from dict', 'from dict']
3.15.0rc1 default ['from descriptor', 'from dict', 'from dict', 'from dict']
3.15.0rc1 free-threaded ['from descriptor', 'from dict', 'from dict', 'from dict']
Expected output is four "from descriptor" values on every version, as
on 3.13.
The first access runs through the general path and honours the
descriptor; the instruction is then specialized and subsequent
executions read m.__dict__["x"]. getattr(m, "x") returns
"from descriptor" every time, since it does not go through the
specialized instruction, so the same expression gives different
answers depending on how it is spelled and how many times it has run.
Expected: a data descriptor on the type takes precedence over the
instance dictionary for a module subclass exactly as it does for any
other class, and as it did on 3.13 and earlier.
Observed on 3.14.7 and 3.15.0rc1, both default and free-threaded
builds; the same code is on main.
Cause
In Python/specialize.c, _Py_Specialize_LoadAttr selects the module
path with
else if (Py_TYPE(owner)->tp_getattro == PyModule_Type.tp_getattro) {
fail = specialize_module_load_attr(owner, instr, name);
}
which any ModuleType subclass that does not override
__getattribute__ or __getattr__ satisfies.
specialize_module_load_attr_lock_held then only inspects the module
dictionary (unicode keys, no __getattr__ entry, the name present,
a keys version); it never looks the name up on the type. The guard of
the emitted LOAD_ATTR_MODULE is likewise only the dict keys version,
so a descriptor added to the type later is not noticed either.
This came in with gh-103951 ("Fast attribute access for module
subclasses", PR #126264, merged 2024-11-15), which relaxed the guard
from PyModule_CheckExact for speed. The discussion there was about
keeping the guard cheap: PyModule_Check was rejected because it
walks the MRO, and the tp_getattro comparison was chosen as a single
pointer compare that admits only types with module attribute
semantics. The stated motivation was the "Customizing module attribute
access" pattern from the data model docs, that is, assigning a
ModuleType subclass to a module's __class__. Neither the issue
comments nor the PR body, review threads or comments mention
descriptors, property, or type version tags, so this looks like an
unintended consequence rather than a decision. A search of the
tracker (LOAD_ATTR_MODULE, module subclass descriptor, ModuleType
subclass property, and similar) found no prior report.
Impact
Any library that assigns a ModuleType subclass to a module's
__class__ in order to intercept attribute access with descriptors
(the documented route for module-level properties and lazy
attributes, per the "Customizing module attribute access" section of
the data model docs) sees the interception disappear after the first
access on 3.14+. The workaround is to define a __getattribute__ on
the subclass that delegates to the base, which gives the type its own
tp_getattro and so avoids the specialization, at the cost of a
Python-level call on every attribute access to that module.
Encountered in wrapture (https://github.com/GrahamDumpleton/wrapture),
which uses exactly this technique to intercept module attribute access;
repro above is reduced from that.
AI Disclaimer
This was a real problem I encountered, but have had AI generate the report for me so more clearly explained. The AI did generate a suggested fix as well, but I am not in a position to evaluate whether it is correct so have not included it. If want AI generated suggested fix then let me know.
CPython versions tested on:
3.14
Operating systems tested on:
macOS
Linked PRs
- gh-156474
- Vorherrschende Sprache
- Python
- Sterne
- 77.2k
- Forks
- 36k
- Ø Merge
- 1 T. 9 Std.
- Gemergte PRs (30 T.)
- 558
Beitragsleitfaden
Erste Schritte
- Lesen Sie das ganze Issue und danach den Beitragsleitfaden des Projekts.
- Schreiben Sie ins Issue, dass Sie es übernehmen — das erspart doppelte Arbeit.
- Forken Sie das Repository und arbeiten Sie in einem Branch.
- Öffnen Sie einen Pull Request, der die Issue-Nummer nennt.
Mehr aus python/cpython
-
docs pending
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 78/100
-
stdlib type-feature
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 78/100
-
stdlib type-feature
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 72/100
-
build type-bug
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 76/100
-
stdlib topic-email type-feature
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 70/100
Ähnliche Issues
-
link-check link-check:sphinx-theme
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 72/100
-
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 65/100
qgis/QGIS-Documentation#11275 ·
-
bug priority:normal ready-for-dev
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 88/100
OpenHands/extensions#626 · 1 Kommentar ·
-
Schwierigkeit 1/5 Unter einer Stunde Anfängerfreundlichkeit 90/100
CSCfi/sd-search-api#39 ·
-
Schwierigkeit 1/5 Unter einer Stunde Anfängerfreundlichkeit 90/100