python / python/cpython

`test_embed` segfaults on BOLT instrumented binaries

Offen
#128,473 1 Kommentar 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen

Dieses Issue hat noch niemand übernommen.

build tests type-crash
Vorherrschende Sprache
Python
Sterne
77.2k
Forks
36k
PR-Kennzahlen ausstehend

Beschreibung

Bug report

Bug description:

Originally reported in https://github.com/astral-sh/python-build-standalone/pull/463

The test_embed suite fails with BOLT instrumented binaries. This requires configuration with the --enable-bolt and --enable-shared flags as described in #128472.

e.g.

LD_LIBRARY_PATH=/home/zb/workspace/cpython ./python -m test --pgo --timeout=
Using random seed: 703977063
0:00:00 load avg: 4.41 Run 44 tests sequentially in a single process
...
0:00:22 load avg: 3.44 [15/44] test_difflib
0:00:23 load avg: 3.44 [16/44] test_embed
test test_embed failed
0:00:51 load avg: 2.76 [17/44] test_float -- test_embed failed (3 failures)
0:00:51 load avg: 2.76 [18/44] test_fstring
0:00:54 load avg: 2.62 [19/44] test_functools
...
Total duration: 2 min 2 sec
Total tests: run=9,524 failures=3 skipped=219
Total test files: run=44/44 failed=1
Result: FAILURE

I believe this occurs on older Python version too, but is a non-fatal error for the test suite — I have not confirmed this yet. On 3.13+, it's a fatal error.

There's a patch disabling these tests at https://github.com/astral-sh/python-build-standalone/pull/469/commits/50c0317689e18bdb51ae459933366a9752837d17 that I can upstream to unblock BOLT for Python 3.13+. These tests are probably not critical for performance.

Additionally, we'll want to debug why these are segfaulting — that's a bit more involved and can happen after.

Additional logs 
Checked 112 modules (34 built-in, 74 shared, 3 n/a on linux-x86_64, 0 disabled, 1 missing, 0 failed on import)
make[2]: Leaving directory '/fast/workspace/cpython'
make[1]: Leaving directory '/fast/workspace/cpython'
make profile-bolt-stamp
make[1]: Entering directory '/fast/workspace/cpython'
# Ensure a pristine, pre-BOLT copy of the binary and no profile data from last run.
for bin in python libpython3.14.so.1.0; do \
  prebolt="${bin}.prebolt"; \
  if [ -e "${prebolt}" ]; then \
    echo "Restoring pre-BOLT binary ${prebolt}"; \
    mv "${bin}.prebolt" "${bin}"; \
  fi; \
  cp "${bin}" "${prebolt}"; \
  rm -f ${bin}.bolt.*.fdata ${bin}.fdata; \
done
# Instrument each binary.
for bin in python libpython3.14.so.1.0; do \
  /usr/bin/llvm-bolt "${bin}" -instrument -instrumentation-file-append-pid -instrumentation-file=/fast/workspace/cpython/${bin}.bolt -o ${bin}.bolt_inst -update-debug-sections -skip-funcs=_PyEval_EvalFrameDefault,sre_ucs1_match/1,sre_ucs2_match/1,sre_ucs4_match/1 ; \
  mv "${bin}.bolt_inst" "${bin}"; \
done
BOLT-INFO: Target architecture: x86_64
BOLT-INFO: BOLT version: 6a0964d75628b15bafd078342120888c0e6d126f
BOLT-INFO: first alloc address is 0x400000
BOLT-INFO: creating new program header table at address 0x600000, offset 0x200000
BOLT-INFO: enabling relocation mode
BOLT-INFO: forcing -jump-tables=move for instrumentation
BOLT-INFO: enabling lite mode
BOLT-INFO: 0 out of 6 functions in the binary (0.0%) have non-empty execution profile
BOLT-INSTRUMENTER: Number of indirect call site descriptors: 2
BOLT-INSTRUMENTER: Number of indirect call target descriptors: 4
BOLT-INSTRUMENTER: Number of function descriptors: 4
BOLT-INSTRUMENTER: Number of branch counters: 1
BOLT-INSTRUMENTER: Number of ST leaf node counters: 4
BOLT-INSTRUMENTER: Number of direct call counters: 0
BOLT-INSTRUMENTER: Total number of counters: 5
BOLT-INSTRUMENTER: Total size of counters: 40 bytes (static alloc memory)
BOLT-INSTRUMENTER: Total size of string table emitted: 47 bytes in file
BOLT-INSTRUMENTER: Total size of descriptors: 356 bytes in file
BOLT-INSTRUMENTER: Profile will be saved to file /fast/workspace/cpython/python.bolt
BOLT-INFO: padding code to 0xa00000 to accommodate hot text
BOLT-INFO: output linked against instrumentation runtime library, lib entry point is 0xa06950
BOLT-INFO: clear procedure is 0xa02390
BOLT-INFO: patched build-id (flipped last bit)
BOLT-INFO: setting _end to 0xa0a0bc
BOLT-INFO: setting _end to 0xa0a0bc
BOLT-INFO: setting __bolt_runtime_start to 0xa06900
BOLT-INFO: setting __bolt_runtime_fini to 0xa06950
BOLT-INFO: setting __hot_start to 0x800000
BOLT-INFO: setting __hot_end to 0x80014c
BOLT-INFO: shared object or position-independent executable detected
BOLT-INFO: Target architecture: x86_64
BOLT-INFO: BOLT version: 6a0964d75628b15bafd078342120888c0e6d126f
BOLT-INFO: first alloc address is 0x0
BOLT-INFO: creating new program header table at address 0x600000, offset 0x600000
BOLT-INFO: enabling relocation mode
BOLT-INFO: forcing -jump-tables=move for instrumentation
BOLT-INFO: enabling lite mode
BOLT-WARNING: Failed to analyze 1380 relocations
BOLT-INFO: 0 out of 7363 functions in the binary (0.0%) have non-empty execution profile
BOLT-INSTRUMENTER: Number of indirect call site descriptors: 1508
BOLT-INSTRUMENTER: Number of indirect call target descriptors: 7281
BOLT-INSTRUMENTER: Number of function descriptors: 7281
BOLT-INSTRUMENTER: Number of branch counters: 82309
BOLT-INSTRUMENTER: Number of ST leaf node counters: 43949
BOLT-INSTRUMENTER: Number of direct call counters: 0
BOLT-INSTRUMENTER: Total number of counters: 126258
BOLT-INSTRUMENTER: Total size of counters: 1010064 bytes (static alloc memory)
BOLT-INSTRUMENTER: Total size of string table emitted: 186842 bytes in file
BOLT-INSTRUMENTER: Total size of descriptors: 8303240 bytes in file
BOLT-INSTRUMENTER: Profile will be saved to file /fast/workspace/cpython/libpython3.14.so.1.0.bolt
BOLT-INFO: 59587 instructions were shortened
BOLT-INFO: removed 161 empty blocks
BOLT-INFO: UCE removed 1623 blocks and 100292 bytes of code
BOLT-INFO: padding code to 0x1000000 to accommodate hot text
BOLT-INFO: output linked against instrumentation runtime library, lib entry point is 0x11bd950
BOLT-INFO: clear procedure is 0x11b9390
BOLT-INFO: patched build-id (flipped last bit)
BOLT-WARNING: [internal-dwarf-error]: unsupported DW_OP encoding.
BOLT-WARNING: [internal-dwarf-error]: unsupported DW_OP encoding.
BOLT-WARNING: [internal-dwarf-error]: unsupported DW_OP encoding.
BOLT-WARNING: [internal-dwarf-error]: unsupported DW_OP encoding.
BOLT-WARNING: [internal-dwarf-error]: unsupported DW_OP encoding.
BOLT-WARNING: [internal-dwarf-error]: unsupported DW_OP encoding.
BOLT-WARNING: [internal-dwarf-error]: unsupported DW_OP encoding.
BOLT-WARNING: [internal-dwarf-error]: unsupported DW_OP encoding.
BOLT-WARNING: [internal-dwarf-error]: unsupported DW_OP encoding.
BOLT-WARNING: [internal-dwarf-error]: unsupported DW_OP encoding.
BOLT-INFO: setting __bolt_runtime_start to 0x11bd900
BOLT-INFO: setting __bolt_runtime_fini to 0x11bd950
BOLT-INFO: setting __hot_start to 0x800000
BOLT-INFO: setting __hot_end to 0xe06596
# Run instrumented binaries to collect data.
LD_LIBRARY_PATH=/home/zb/workspace/cpython ./python -m test --pgo --timeout=
Using random seed: 703977063
0:00:00 load avg: 4.41 Run 44 tests sequentially in a single process
0:00:00 load avg: 4.41 [ 1/44] test_array
0:00:01 load avg: 4.41 [ 2/44] test_base64
0:00:03 load avg: 4.41 [ 3/44] test_binascii
0:00:04 load avg: 4.41 [ 4/44] test_binop
0:00:04 load avg: 4.41 [ 5/44] test_bisect
0:00:04 load avg: 4.41 [ 6/44] test_bytes
0:00:08 load avg: 4.14 [ 7/44] test_bz2
0:00:09 load avg: 3.89 [ 8/44] test_cmath
0:00:09 load avg: 3.89 [ 9/44] test_codecs
0:00:11 load avg: 3.89 [10/44] test_collections
0:00:12 load avg: 3.89 [11/44] test_complex
0:00:13 load avg: 3.89 [12/44] test_dataclasses
0:00:13 load avg: 3.89 [13/44] test_datetime
0:00:19 load avg: 3.65 [14/44] test_decimal
0:00:22 load avg: 3.44 [15/44] test_difflib
0:00:23 load avg: 3.44 [16/44] test_embed
test test_embed failed
0:00:51 load avg: 2.76 [17/44] test_float -- test_embed failed (3 failures)
0:00:51 load avg: 2.76 [18/44] test_fstring
0:00:54 load avg: 2.62 [19/44] test_functools
0:00:56 load avg: 2.62 [20/44] test_generators
0:00:57 load avg: 2.62 [21/44] test_hashlib
0:00:57 load avg: 2.62 [22/44] test_heapq
0:00:58 load avg: 2.62 [23/44] test_int
0:01:00 load avg: 2.49 [24/44] test_itertools
0:01:05 load avg: 2.37 [25/44] test_json
0:01:18 load avg: 2.47 [26/44] test_long
0:01:20 load avg: 2.35 [27/44] test_lzma
0:01:21 load avg: 2.35 [28/44] test_math
0:01:24 load avg: 2.35 [29/44] test_memoryview
0:01:25 load avg: 2.24 [30/44] test_operator
0:01:25 load avg: 2.24 [31/44] test_ordered_dict
0:01:28 load avg: 2.24 [32/44] test_patma
0:01:29 load avg: 2.24 [33/44] test_pickle
0:01:34 load avg: 2.14 [34/44] test_pprint
0:01:34 load avg: 2.05 [35/44] test_re
0:01:35 load avg: 2.05 [36/44] test_set
0:01:41 load avg: 1.97 [37/44] test_sqlite3
0:01:44 load avg: 1.97 [38/44] test_statistics
0:01:49 load avg: 1.89 [39/44] test_str
0:01:52 load avg: 1.82 [40/44] test_struct
0:01:53 load avg: 1.82 [41/44] test_tabnanny
0:01:56 load avg: 1.91 [42/44] test_time
0:01:58 load avg: 1.91 [43/44] test_xml_etree
0:02:00 load avg: 1.84 [44/44] test_xml_etree_c
Total duration: 2 min 2 sec
Total tests: run=9,524 failures=3 skipped=219
Total test files: run=44/44 failed=1
Result: FAILURE
make[1]: *** [Makefile:969: profile-bolt-stamp] Error 2
make[1]: Leaving directory '/fast/workspace/cpython'
CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Linked PRs
  • gh-128474

Beitragsleitfaden

Beitragsleitfaden öffnen

Erste Schritte

  1. Lies das ganze Issue und danach den Beitragsleitfaden des Projekts.
  2. Schreib ins Issue, dass du es übernimmst — das erspart doppelte Arbeit.
  3. Forke das Repository und arbeite in einem Branch.
  4. Öffne einen Pull Request, der die Issue-Nummer nennt.

Rechercherichtung

Überprüfe gh-128474 und die Fehler in test_embed unter --enable-bolt und --enable-shared. Reproduziere das Problem mit dem dokumentierten LD_LIBRARY_PATH=... ./python -m test --pgo --timeout=-Befehl und untersuche anschließend die test_embed-Suite und ihre drei Fehler. Erledigt ist die Aufgabe, wenn der mit BOLT instrumentierte Testlauf nicht mehr mit einem Segfault abbricht oder für diese Tests eine vereinbarte, getestete Behandlung vorhanden ist.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Bewertung

Tech-Stack
python
Bereich
build-system, testing-qa
Issue-Typ
Bug
Schwierigkeit
4/5
Geschätzter Aufwand
3-5 Tage
Aktivitätsstatus
Veraltet
Klarheit
Muss geklärt werden
Anfängerfreundlichkeit
25/100

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.