Some DTrace probes are broken in 3.11

Crash report

I've been using eBPF with the static markers in Python 3.10 and wanted to try out 3.11 now that it is released.

But it seems that the function__entry and line markers are broken in 3.11.

The function__entry probe simply does not fire into the eBPF program.
The line probe crashes the interpreter.

I'm using BCC to load the eBPF program like this:

#!/usr/bin/python3

import argparse
from bcc import BPF, USDT

parser = argparse.ArgumentParser()
parser.add_argument("pid", type=int)
args = parser.parse_args()

program = """
int trace_entry(struct pt_regs *ctx) {
    bpf_trace_printk("Entry:");

    return 0;
}

int trace_return(struct pt_regs *ctx) {
    bpf_trace_printk("Return:");

    return 0;
}

int trace_line(struct pt_regs *ctx) {
    bpf_trace_printk("Line:");

    return 0;
}
"""

usdt = USDT(pid=args.pid)
usdt.enable_probe_or_bail("python:function__entry", 'trace_entry')
usdt.enable_probe_or_bail("python:function__return", 'trace_return')
usdt.enable_probe_or_bail("python:line", 'trace_line')

bpf = BPF(text=program, usdt_contexts=[usdt] if usdt else [], debug=0)

try:
    bpf.trace_print()
except KeyboardInterrupt:
    exit()

If I don't enable the line probe this is the output:

❯ sudo ./ebpf-test.py 80715
b'          python-80715   [001] d...1  6539.145626: bpf_trace_printk: Return:'

The entry message is never printed.

Error messages

With the line probe enabled, this is the crash error:

❯ ./python
Python 3.11.0+ (heads/3.11:57dd11038f, Oct 31 2022, 10:30:28) [GCC 11.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> print("Hello world!")
python: Python/ceval.c:5620: _PyEval_EvalFrameDefault: Assertion `cframe.use_tracing' failed.
[1]    80398 IOT instruction (core dumped)  ./python

Expected result

Running the same test in 3.10.6 I get this result, as expected:

❯ python3                      
Python 3.10.6 (main, Aug 10 2022, 11:40:04) [GCC 11.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> print("Hello world!")
Hello world!

❯ sudo ./ebpf-test.py 81962
b'         python3-81962   [003] d...1  6907.857690: bpf_trace_printk: Entry:'
b'         python3-81962   [003] d...1  6907.857710: bpf_trace_printk: Line:'
b'         python3-81962   [003] d...1  6907.857765: bpf_trace_printk: Return:'

Your environment

I used the head of the 3.11 branch with ./configure --with-dtrace --with-pydebug
On Ubuntu 22.04.

Linked PRs

• gh-125019
• gh-139334
• gh-142397
• gh-151122
• gh-151235
• gh-152239
• gh-152300
• gh-152301
• gh-152302
• gh-152345
• gh-152891
• gh-152893
• gh-152900
• gh-152901
• gh-153372
• gh-153459
• gh-154803
• gh-155803
• gh-156424

Commencez par l’assertion dans Python/ceval.c:5620 et les sondes DTrace nommées function__entry, function__return et line. Reproduisez le comportement à l’aide du script eBPF BCC fourni sur un build configuré avec --with-dtrace, puis comparez les résultats de 3.11 avec ceux de 3.10. Le travail est considéré comme terminé lorsque les sondes d’entrée et de ligne se déclenchent sans faire planter l’interpréteur, tandis que la sonde de retour continue de fonctionner.