Triple dispatching our way into a smaller interpreter

Feature or enhancement

Proposal:

This is orthogonal to https://github.com/python/cpython/issues/148506. However, the problem remains the same. I'll repeat it again here: we have hit the limit of the computed goto/switch-case interpreter size such that we are seeing compiler bugs in all three: MSVC, Clang, and GCC. (I just fixed another compiler bug in Clang 22 due to interpreter size last week). The tail calling interpreter is the long-term solution. In the short term, we need another one. It is imperative that we reduce the size of the interpreter.

Instrumentation takes up 21 opcodes. This presents an opportunity for significant interpreter size reductions and recovery of opcodes. With this, we can remove all instrumented instructions. The INSTRUMENTED_X instructions become pseudo instructions with oparg > 255 . We store the instrumented pseudo-instruction in the instrumentation tools in get_tools_for_instruction instead of the bytecode, so that we can exceed the 255 limit.

To achieve instrumentation, we just swap out the interpreter dispatch table. The key observation is to repurpose the current TRACE_RECORD instruction to a VARIABLE_DISPATCH operation and change that single instruction to use call threading. This call threading instruction will support both JIT and instrumentation modes. We also move all instrumentation functions to small helper functions automatically using the cases generator. This is a small change, as we already refactored the interpreter to almost support call-threading due to the tail calling interpreter work, where each opcode is implemented as a function.

// For instrumentation
static void *instrumented_dispatch_table[256] = {
    [FOR_ITER] = &&VARIABLE_DISPATCH,
}

static funcptr instrumented_targets_table[256] = {
    [FOR_ITER] = &_CALL_FOR_ITER,
    [INSTRUMENTED_FOR_ITER] = &_CALL_INSTRUMENTED_FOR_ITER,
}

// For JIT
static void *tracing_targets_table[256] = {
    [FOR_ITER] = &&VARIABLE_DISPATCH,
}

inst(VARIABLE_DISPATCH, (--)) {
    next_instr = this_instr;
    if (dispatch_table_var == instrumented_dispatch_table) {
        if (HAS_INSTRUMENTED_OPCODES[opcode]) {
            opcode = get_instrumented_opcode(inst);
            _CALL_ARGS = instrumented_targets_table[opcode](_CALL_ARGS);
            DISPATCH();
        }
        else {
            DISPATCH_NON_INSTRUMENTED();
         }
    }
    else {
        assert(dispatch_table_var == tracing_targets_table);
        // Do what _TRACE_RECORD currently does in the JIT.
    }
}

This will remove all instrumented opcodes from the main interpreter. Runtime instrumentation might be slower, but still faster than the old days of sys.settrace(). However instrumentation pauses will be significantly lower, as we won't have to scan bytecode to instrument, instead we just check the current tools and swap the dispatch table. Finally, the base interpreter should be faster too.

Has this already been discussed elsewhere?

No response given

Links to previous discussion of this feature:

No response

首先阅读 TRACE_RECORD、VARIABLE_DISPATCH、get_tools_for_instruction 附近的解释器分发逻辑，以及 proposal 中提到的 cases generator。跟踪 instrumentation 和 JIT 分发表当前是如何工作的。完成的标准是从主解释器中移除经过 instrumentation 的指令，同时保留 instrumentation 和 JIT 的行为，并减小解释器的大小。