python / python/cpython

Avoid unnecessary NotShareableError creation in XIData FULL_FALLBACK

Offen
#148,223 0 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen

Dieses Issue hat noch niemand übernommen.

interpreter-core type-feature
Vorherrschende Sprache
Python
Sterne
77.2k
Forks
35.9k
PR-Merge-Kennzahlen
PR-Kennzahlen ausstehend

Beschreibung

Proposal:
Motivation

The pickle fallback path in _PyObject_GetXIData() does unnecessary work on successful transfers of types that are not in the XIData registry, creating and discarding a NotShareableError that user code never sees.

This is a follow-up to gh-148072 / GH-148125, which cached pickle.dumps and pickle.loads per interpreter. That change removed module lookup overhead, but there is still avoidable cost in the fallback path itself.

This extra exception work adds measurable overhead to cross-interpreter transfers that rely on pickle fallback.

Approach

Check the XIData registry first with lookup_getdata() before calling _get_xidata().

  • If the type is registered, keep using the existing path unchanged.
  • If the type is not registered, skip _get_xidata() and go directly to the function/pickle fallback chain.

On total failure, raise the same NotShareableError as the existing code path, so external behavior stays the same.

Benchmark results

This removes unnecessary exception creation on a hot fallback path and gives a measurable speedup for transfers that rely on pickle fallback.

In local benchmarks on Apple M2, combined with GH-148125, this improved round-trip transfer times for mutable objects such as list and dict, especially for larger payloads.

Large-payload benchmark results (500 iterations, Apple M2)

Baseline is upstream/main after GH-148125 (pickle cache). "After" includes this change.

Data Before After Speedup
list [1 x 100KB] 1.19ms 0.12ms 9.9x
list [1 x 1MB] 10.26ms 0.20ms 51x
list [10 x 100KB] 9.99ms 0.10ms 100x
dict {10 x 100KB} 10.41ms 0.26ms 40x
Related
  • gh-148072 — Predecessor: pickle module caching (merged as GH-148125)
  • gh-124694 — InterpreterPoolExecutor addition
Validation

Local testing passed:

  • ./python -m test test_interpreters
  • ./python -m test test_concurrent_futures
Has this already been discussed elsewhere?

This is a minor feature, which does not need previous discussion elsewhere

Links to previous discussion of this feature:
Linked PRs
  • gh-148224

Beitragsleitfaden

Beitragsleitfaden öffnen

Erste Schritte

  1. Lies das ganze Issue und danach den Beitragsleitfaden des Projekts.
  2. Schreib ins Issue, dass du es übernimmst — das erspart doppelte Arbeit.
  3. Forke das Repository und arbeite in einem Branch.
  4. Öffne einen Pull Request, der die Issue-Nummer nennt.

Rechercherichtung

Beginne mit _PyObject_GetXIData() und verfolge lookup_getdata() und _get_xidata() im Pickle-Fallback-Pfad. Prüfe den verlinkten PR gh-148224 und führe anschließend ./python -m test test_interpreters und ./python -m test test_concurrent_futures aus; abgeschlossen ist die Arbeit, wenn das bestehende Verhalten von NotShareableError beibehalten und die unnötige Erstellung von Exceptions vermieden wird.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Bewertung

Tech-Stack
python
Bereich
backend, performance
Issue-Typ
Refactoring
Schwierigkeit
4/5
Geschätzter Aufwand
3-5 Tage
Aktivitätsstatus
Veraltet
Klarheit
Größtenteils klar
Anfängerfreundlichkeit
25/100

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.